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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI)  is  developing  methods  to  aid  in  the 
successful  integration  of  soldiers  and  their  equipment.  Methods 
are  only  as  effective  as  their  originating  concepts.  As  part  of 
this  effort,  ARI  has  defined  the  requirements  for  writing  a 
series  of  detailed  concept  papers.  These  papers  will  serve  as 
conceptual  underpinnings  and  as  the  first  stage  of  the  program 
for  building  successful  integration  methods. 

This  monograph  describes  three  alternative  concepts  for 
building  a  method  to  aid  in  predicting  probable  training  for  a 
developing  weapon  system.  The  resulting  description  of  training 
will  serve  to  constrain  the  design  of  the  weapon  system.  These 
concepts  will  serve  both  as  the  focus  of  current  system  building 
and  as  a  seedbed  for  future  concepts. 


EDGAR  M.  JOHNSON 
Technical  Director 
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MANPRINT  METHODS  MONOGRAPH:  AIDING  THE  DEVELOPMENT  OF  TRAINING 
CONSTRAINTS 


EXECUTIVE  SUMMARY 


The  U.S.  Array  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI)  is  conducting  a  program  to  develop  methods 
to  aid  in  successfully  integrating  available  operations  and 
maintenance  personnel  with  hardware  and  software  as  part  of  the 
general  MANPRINT  process.  To  do  this,  ARI  defined  and  produced 
requirements  for  six  classes  of  aiding  methods.  In  the  first 
phase  t^is  effort,  three  alternative  concepts  were  developed 
for  each  of  the  cix  classes  of  aiding  methods. 

This  monograph  consists  of  three  concept  papers.  These 
papers  describe  alternative  concepts  for  aiding  the  process  of 
predicting  probable  weapon  system  training  so  as  to  constrain  the 
design  of  that  weapon  system. 

Ultimately,  the  ARI  study  advisory  group  selected  the 
concept  proposed  by  Applied  Science  Associates  for 
implementation.  However,  each  of  the  three  concepts  is  the 
result  of  serious  thinking  about  how  to  deal  with  training 
prediction.  As  such,  all  three  concept  papers  may  prove  of 
considerable  value  for  future  research  in  this  area. 
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AIDING  THE  DEVELOPMENT  OF  TRAINING  CONSTRAINTS 


Introduction 

The  U.S.  Army  Research  Institute  (ARI)  is  conducting  a 
research  program  to  develop  methods  to  aid  in  successfully 
integrating  available  operations  and  maintenance  personnel 
with  hardware  and  software  as  part  of  the  general  MANPRINT 
process.  To  do  this,  ARI  defined  and  produced  requirement s 
for  six  classes  of  aiding  methods.  The  first  four  of  these 
methods  will  aid  the  integration  process  by  developing 
information  that  will  be  used  as  system  design  constraints. 
This  information  will  be  used  in  requirements  documents  and 
will  be  provided  to  potential  design  organizations.  The 
last  two  of  these  methods  will  aid  the  integration  process 
by  providing  mechanisms  to  evaluate  system  designs. 

This  monograph  consists  of  three  papers  on  a  common 
subject--How  to  aid  in  the  development  of  predictions  of 
training  that  will  constrain  system  design.  Each  of  these 
three  papers  presents  a  concept  for  building  an  aiding 
method  for  making  these  predictions.  These  methods  would 
all  be  software  based  and  provide  aid  without  significantly 
raising  their  user's  workload.  All  concepts  were  generated 
in  response  to  the  same  Army  requirement.  To  fully 
understand  these  concepts,  one  must  first  understand  the 
context  in  which  they  were  developed. 

The  Army  acquires  systems  as  a  mechanism  for  obtaining 
needed  performance.  The  hardware  and  software  components  of 
most  Army  systems  are  operated  and  maintained  by  soldiers. 
Therefore,  soldiers  are  components  of  those  systems.  The 
performance  and  availability  levels  of  systems  are  directly 
related  to  the  performance  of  the  trained  soldiers  who 
operate,  maintain,  and  support  their  hardware  and  software. 

The  Army  must  assume  that  all  trained  operator  and 
maintenance  crews  assigned  to  given  pieces  of  hardi.are  will 
produce  at  least  minimally  adequate  system  performance  and 
availability  so  that  commanders  will  have  a  basis  for  making 
battlefield  decisions.  Training  affects  soldier  performance 
and  therefore  how  those  soldiers  interact  with  system 
hardware  and  software.  That  being  the  case,  the  following 
two  approaches  to  system  design  should  be  considered. 

1.  Hardware  and  software  components  should  be  selected  or 
designed  so  that  they  can  be  integrated  successfully  with 
those  training  characteristics  that  are  likely  to  exist  when 
the  system  is  fielded.  In  this  approach,  available  training 
becomes  a  constraint  and  therefore  a  driver  on  system 
design. 
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2 •  Training  should  be  selected  or  designed  so  as  to  lead  to 
successful  performance  when  its  recipients  operate  or 
maintain  specific  systems.  In  this  approach,  system  design 
becomes  a  constraint  and  therefore  a  driver  on  training 
design . 

This  approach  is  somewhat  unusual  in  the  training 
world.  Typically,  what  would  be  developed  prior  to  system 
design  is  a  method  for  early  determination  of  training 
requirements  rather  than  constraints.  The  logic  behind  the 
de§ire  to  develop  a  training  constraints  aid  is  based  on 
the  observation  that  when  there  is  a  natural,  sequential 
flow  to  events,  following  that  sequence  leads  to  cost- 
effective  output.  Attempting  to  go  contrary  to  such  a  flow 
usually  results  in  low  effectiveness,  high  cost,  or  both. 

Systems  ought  to  be  acquired,  and  therefore  designed, 
to  produce  mission  success.  All  other  aspects  of  systems 
acquisition,  including  training,  must  lead  to  this  desired 
end.  The  training  requirements  approach  must  be  based  upon 
a  reasonably  detailed  understanding  of  what  the  tasks  will 
be  that  will  result  in  mission  success  and  how  those  tasks 
will  be  performed.  Some  form  of  task  information  can  be 
developed  prior  to  system  design,  but  the  knowledge  of  how 
tasks  are  to  be  performed  is  a  direct  function  of  the  system 
design.  That  is,  in  the  training  requirements  approach, 
training  design  is  driven  by  system  design.  One  might 
develop  hypothetical  designs,  and  determine  their  training 
requirements,  but  this  is  not  cost-effective  if  there  is  a 
significant  amount  of  time  available  for  requirements 
determination  following  the  development  of  the  real,  initial 
system  design.  Usually,  there  is  a  large  amount  of  time 
between  initial  system  design  development  and  fielding,  and 
such  an  amount  should  be  sufficient  for  determining  training 
requirements . 

That  being  the  case,  to  make  a  cost-effective 
impression  on  system  design,  one  must  use  the  constraints 
approach  and  attempt  to  drive  system  design  with  probable 
training.  Once  an  initial  system  design  has  been  made,  it 
can  be  used  to  determine  required  training.  When  this  has 
been  done,  the  differences  between  the  required  and  probable 
training  can  serve  as  the  basis  for  a  training  analysis  of 
costs  versus  effects  of  making  the  required  training 
changes . 

This  monograph  was  driven  by  a  requirement  to  develop 
alternate  concepts  for  predicting  probable  training.  The 
three  concepts  presented  were  written  by  personnel  from 
Applied  Science  Associates  Inc.  (ASA),  Horizons  Technology, 
Inc.  (HTI),  and  Dynamics  Research  Corp.  (DRC).  Eventually, 
ARI  chose  to  complete  the  development  of  ASA's  concept. 
However,  all  three  concepts  have  considerable  merit  and  are 
quite  diverse  in  approach. 
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The  ASA  concept  is  based  upon  the  notion  that  the  major 
dimension  for  predicting  training  is  inertia.  That  is,  the 
training  for  new  or  hypothetical  systems  is  likely  to  be 
similar  to  training  for  existing  systems  regardless  of  the 
adequacy  of  the  previous  training  or  resource  requirements 
of  future  training.  Based  on  this  approach,  the  real 
problem  to  be  solved  is  how  to  compare  hypothetical  to 
existing  systems  so  as  to  determine  what  existing  systems 
should  serve  as  sources  of  training  information. 

The  HTI  concept  is  based  upon  the  development  of  two 
methods,  one  for  determining  probable  training  and  one  for 
determining  required  training.  The  method  for  determining 
probable  training  is  conceptually  similar  to  the  ASA  notion. 
The  method  for  determining  required  training  is  based  upon 
structuring  the  detailed  views  of  subject  matter  experts. 

The  DRC  concept  is  based  upon  the  notion  that  the  major 
dimension  for  predicting  training  is  the  current 
availability  of  training  assets.  This  concept  assumes  that 
training  is  driven  by  available  numbers  of  training  assets 
including  dollars,  and  the  key  to  predicting  it  is  to 
determine  what  those  assets  will  be. 

The  three  concept  papers  in  this  monograph  have  been 
paginated  as  El,  E2 ,  and  E3  to  delineate  them  clearly. 
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CONCEPT  FOR  A  MANPRINT  TRAINING  CHARACTERISTICS 
ESTIMATION  AID  (PRODUCT  FOUR) 


SECTION  ONE 

INTRODUCTION  AND  OVERVIEW 


Background 


The  U.S.  Array  Research  Institute  (ARI)  has  been,  and  will  continue 
to  be,  a  key  player  in  the  Army's  MANPower  Requirement  INTegration 
(MANPRINT)  initiative.  The  purpose  of  the  MANPRINT  initiative  is  to 
ensure  that  continuous  attention  and  scrutiny  is  applied  to  Manpower, 
Personnel,  Training,  Human  Factors  Engineering,  System  Safety,  and 
Health  Hazards  Assessment  factors  during  system  acquisition.  This 
attention  is  to  assure  that  newly  acquired  Army  systems  will  be  able  to 
meet  their  mission  objectives  with  the  personnel  and  training  that  the 
Array  can  provide.  A  principal  goal  of  MANPRINT  is  to  influence  and 
constrain  the  systems  design  process  so  that  only  the  people  that  the 
Army  has  available  will  be  required  to  operate  and  maintain  new 
systems.  Too  frequently  in  the  past,  the  opposite  has  been  true.  This 
has  led  to  situations  where  systems  demand  unreasonably  capable  or 
numerous  people  for  operations  and  maintenance. 

To  rationally  and  effectively  constrain  the  system  design  process 
so  that  MANPRINT  goals  are  met,  the  Array  must  provide  system  designers 
with  defensible  MANPRINT  parameters  which  their  designs  must  meet  in 
order  to  be  deemed  acceptable.  Currently,  such  parameters  are  based 
principally  on  expert  judgments  provided  by  specialists  in  various 
disciplines  (notably  human  factors  engineering  and  systems  analysis) 
and  by  military  Subject  Matter  Experts  (SMEs).  A  significant  problem 
with  the  use  of  expert  judgments  is  that  opinions  and  estimates  differ 
among  experts.  This  can  sometimes  leave  the  parameters  supplied  to 
designers  open  to  question.  A  further  problem  with  expert  judgments  is 
that  the  process  of  making  such  judgments  is  not  systematized,  and  is 
therefore  not  inherently  auditable  or  defensible.  Finally,  expert 
judgments  vary  widely  in  the  data  considered  and  the  manner  in  which 
judgmental  rules  are  applied.  This  sometimes  makes  it  uncertain  that 
all  relevant  aspects  of  issues  and  their  interactions  have  been 
considered . 

In  addition  to  problems  with  expert  judgment  as  a  basis  for 
MANPRINT  constraints  is  the  fact  that  many  of  the  constraints  must  be 
established  very  early  in  the  acquisition  cycle.  This  means  that 
estimates  are  required  prior  to  the  availability  of  analytically-based 
data  on  the  system  to  be  developed.  Frequently,  only  data  on  the 
system's  mission  and  desired  characteristics  are  available  at  this 
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time.  Techniques  for  projecting  even  preliminary  MANPRINT  requirements 
during  pre-concept  and  concept  development  are  not  well  formalized. 

The  focus  of  the  overall  effort  of  which  this  paper  i6  a  part  is 
to  develop  the  beginnings  of  a  "MANPRINT  estimation  toolkit."  Some  of 
the  tools  to  be  developed  will  he  used  to  identify  system  MANPRINT 
constraints  early  in  the  acquisition  cycle.  Other  tools  will  be  used 
to  evaluate  the  extent  to  which  system  designs  comply  with  MANPRINT 
requirements  and  constraints  after  the  system  is  designed.  Under  the 
present  ARI  effort,  six  discrete  products  will  be  produced  for  use  in 
implementing  the  MANPRINT  initiative.  This  paper  presents  a  concept 
for  Product  Four  of  the  six  products.  The  goal  of  this  Product  is  to 
provide  a  means  of  identifying  the  types  and  amounts  of  training  which 
are  likely  to  be  provided  by  the  Army  for  a  new  system.  This 
information  can  serve  as  a  basis  for  developing  training  character¬ 
istics  constraints  for  a  new  system.  The  remainder  of  this  paper 
details  a  concept  for  development  of  this  Product. 


Product  Philosophy 


The  overall  approach  which  Applied  Science  Associates,  Inc.  and 
Science  Applications  International  Corporation  (ASA/SAIC)  have  adopted 
for  Product  Four  is  an  extension,  with  great  modification,  of  the 
Comparison-Based  Prediction  (CBP)  technique.  This  methodology  was 
pioneered  by  Tetraeyer  (1976)  and  has  been  further  refined  by  Klein  and 
Associates  in  a  series  of  efforts  for  ARI  and  the  U.S.  Air  Force  (Klein 
and  Weitzenfeld,  1982;  Weitzenfeld  and  Klein,  1982;  Klein  and  John, 
1984;  Klein,  Gordon,  Palmisano,  and  Mirabella,  1985;  Klein,  1985). 

This  technique  relies  on  using  experience  with  characteristics  of 
existing  systems  to  project  similar  characteristics  of  contemplated 
systems.  For  example,  cost  and  effectiveness  data  for  existing 
training  devices  versus  actual  equipment  can  generate  implications  for 
a  projected  training  system.  Naturally,  some  modifications  to  the  data 
on  the  existing  system  must  be  made.  It  must  be  emphasized  at  this 
point  that  Product  Four  is  not  conventional  CBP.  It  is  a  much  more 
powerful  technique  which  shares  some  general  philosophy  with  existing 
CBP  methods. 

The  premise  behind  the  choice  of  this  approach  is  the  avai  lability 
of  data  on  the  characteristics  of  training  provided  by  the  Army  for 
many  types  of  systems.  It  is  assumed  that  what  the  Array  presently  does 
in  the  way  of  training  is  a  reasonably  valid  basis  for  estimating  what 
will  be  done  for  training  for  new  systems.  Short  of  radical 
innovations  in  training  methods  or  system  interfaces  with  human 
operators  and  maintainers,  this  is  a  reasonable  assumption.  It  is  also 
reasonable  to  assume  that  any  such  changes  will  be  gradual,  rather  than 
revolutionary.  Thus,  what  is  known  about  the  characteristics  of 
training  systems  that  now  exist  will  be  used  to  project  what  training 
systems  for  new-ac qu  is i t ion  systems  will  be  like. 
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Techniques  developed  to  date  have  some  shortcomings  as  tools  for 
the  MANPRINT  analyst.  Current  CBP  techniques  rely  on  expert  judgment 
(normally  provided  by  SMEs)  to  identify  and  characterize  the 
differences  between  existing  and  projected  systems.  These  differences 
are  then  used  to  prepare  modified  estimates  for  the  projected  system 
based  on  actual  data  for  the  comparison  system(s)  which  are  used. 

Expert  judgment  has  proven  reliable  as  a  foundation  for  the 
conventional  CBP  methodology,  but  has  some  limitations  for  use  in 
MANPRINT  estimation  methods.  First,  gathering  expert  opinion  is  a 
time-consuming  and  often  expensive  process.  A  constraint  upon  this 
Product  is  that  no  new  organizations  or  significant  increase  in 
workload  must  be  required  for  the  Product  to  be  exercised.  Thus,  an 
approach  which  requires  significant  amounts  of  consultation  with  SMEs 
to  derive  comparison  estimates  is  not  acceptable. 

A  second  problem  with  expert  judgment  is  mediation:  reconciling 
the  differences  in  judgments  of  two  or  more  experts  in  a  defensible 
way.  This  is  normally  performed  by  the  analyst.  However,  the  users  of 
this  Product  are  not  expected  to  be  trained  predictive  analysts.  The 
principal  users  of  this  Product  are  analysts  in  the  Concepts  and 
Studies  Branches  of  the  Directorates  of  Combat  Developments  at  system 
proponent  schools,  or  Soldier  Support  Center  (SSC)  analysts.  Such 
personnel  neither  have  such  expertise  nor  can  be  trained  in 
conventional  CBP  methods  in  order  to  exercise  this  Product. 

In  examining  the  processes  involved  in  the  CBP  methodology, 
however,  we  believe  that  the  shortcomings  of  existing  CBP  techniques 
can  be  overcome.  Our  approach  to  overcoming  these  shortcomings  relies 
on  two  observations. 

First,  the  role  of  the  SMZ  as  an  expert  judge  is  primarily  one  of 
applying  definable  rules  and  criteria  to  identify  the  extent  of  differ¬ 
ences  between  some  comparison  system  and  the  target  (new)  system.  When 
developing  comparison  cases  for  characteristics  such  as  reliability  or 
availability,  relatively  fine  judgments  about  the  differences  between 
comparison  systems  and  the  target  system  are  required.  Since  concern 
here  is  with  training  systems,  however,  it  is  likely  that  a  far  greater 
tolerance  in  comparability  will  yield  acceptable,  usable  estimates.  We 
believe  that  it  is  possible  to  capture  and  utilize  information  on 
systems  and  subsystems  performance  and  capabilities,  along  with  general 
dimensions  and  tolerances  of  comparison  supplied  by  SMEs.  This  inform¬ 
ation  will  be  used  to  develop  a  rule  structure  which  allows  the  auto¬ 
mated  development  of  composite  comparison  cases  ,  given  input  of  the 
projected  characteristics  of  the  "new"  system  and  its  component 
subsystems.  In  other  words,  comparison  cases  can  be  developed  from  a 
pool  of  actual  cases.  The  training  system  characteristics  of  the 
projected  system  can  then  be  projected  from  the  training  system 
characteristics  of  the  systems  that  make  up  the  composite  comparison 
case . 


Second,  the  analyst's  role  can  be  largely  automated.  Since 
estimates  for  a  consistent  class  of  characteristics  (those  of  the 
training  system)  are  to  be  developed,  the  guidance  and  structure 
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typically  provided  by  the  analyst  to  develop  composite  comparison  cases 
can  be  generated  once  and  applied  many  times.  A  rule  structure  for  the 
development  of  composite  comparison  cases  can  therefore  be  developed 
and  applied.  Sucn  a  rule  structure  will  provide  predictions  that  will 
be  acceptable  and  defensible  for  the  vast  majority  of  estimates.  A 
second  rule  structure  will  be  developed  to  generate  training  time  and 
some  training  resources  estimates  (e.g.,  training  device  requirements). 
These  estimates  will  be  based  on  charac teris t ics  of  the  training 
systems  for  material  systems  that  make  up  the  comparison  case.  These 
rule  structures  are  the  principal  reason  that  this  technique  should  not 
be  thought  of  ao  CBP.  The  CBP  technique  relies  on  specific  comparison 
cases.  This  technique  relies  on  composite  comparison  cases  for  making 
its  predictions. 

Equipping  the  rule  structures  which  comprise  the  "engines"  of  this 
Product  with  defaults,  and  providing  templates  to  support  input  of  data 
on  the  system  for  which  estimates  are  required  will  support  rapid,  easy 
generation  oi  training  estimates.  The  robustness  of  the  rule 
structures  themselves  will  support  the  development  of  more  refined  or 
sophisticated  estimates,  when  more  detailed  data  are  available. 


Output  Requirements 


In  order  to  use  the  proposed  methodology  as  the  foundation  for 
this  Product,  it  is  necessary  to  specifically  identify  the  types  of 
information  that  we  wish  to  estimate.  The  information  to  be  produced 
by  the  Product  should  be  constrained  by  two  factors: 

1.  The  existence  of  data  for  the  development  of  composite 
comparison  cases  and  descriptive  baseline  data  for  training 
system  characteristics. 

2.  The  characteristics  of  the  users  of  the  outputs  of  the 
Product  and  the  information  that  they  can  most  effectively 
utilize  in  making  design  decisions  that  influence  the 
training  characteristics  of  the  system. 

As  discussed  in  later  sections  of  this  paper,  data  to  develop  the 
necessary  databases  exist  or  will  be  derived  during  the  Product 
development  and  evaluation  period.  The  critical  question  now  becomes: 
"Who  are  the  users  of  Product  outputs,  and  what  are  their  character¬ 
istics  that  should  constrain  the  information  provided  to  them"? 

The  principal  expected  users  of  Product  Four  information  are  the 
system  designers  who  are  responsible  for  developing  the  Soldier/System 
Interface  (SSI).  The  characteristics  of  the  SSI  are  principal  drivers 
of  training  needs  and  requirements,  for  most  or  all  tasks  associated 
with  a  system.  This  is  true  for  all  operator  tasks,  and  for  many 
maintainer  tasks  as  well. 
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What  are  Che  characteristics  of  the  users  that  are  relevant  to  the 
information  that  Product  Four  should  provide?  First,  typical  interface 
designers  do  not  understand  the  impact  of  design  choices  on  training 
requirements  and  training  system  characteristics.  They  generally 
cannot  project  or  estimate  whether  design  choices  will  impact  training, 
or  whether  impacts  may  be  positive  or  negative.  Therefore,  desired 
training  system  characteristics  alone  will  not  be  of  much  use  to  such 
users.  Such  data  may,  however,  be  useful  to  the  acquisition  community 
(a  second  class  of  users)  in  assessing  potential  or  desirable  training 
system  characteristics.  These  data  may  also  be  useful  for  evaluating 
the  extent  to  which  the  ultimate  system  designs  meet  MANPRINT  training 
criteria. 

Given  that  training  system  characteristics  data  alone  are  not  of 
much  use  to  the  designer,  other  classes  of  data  which  might  prove 
useful  were  explored.  Since  training  needs  are  driven  by  character¬ 
istics  of  the  SSI,  information  that  could  influence  the  choices  made  in 
SSI  design  would  be  useful  to  present  to  the  designer-user.  Training 
system  characteristics  information  also  has  uses  in  the  system 
acquisition  process,  and  Product  Four  must  produce  such  information,  as 
well. 


The  next  requirement  is  to  identify  specific  information  that  will 
be  of  use  in  constraining  SSI  design  choices.  Data  which  are  likely  to 
influence  design  choices  in  the  desired  ways  are  human  factors 
engineering  and  other  design  principles.  When  such  principles  are 
effectively  applied,  they  tend  to  simplify  and  rationalize  the  SSI. 

This  facilitates  learning  operations  and  maintenance  tasks  for  a 
system,  which  can  tend  to  reduce  training  requirements.  However, 
simply  providing  a  "human  factors  bibie"  for  the  SSI  designer  will  not 
be  sufficient  to  realize  MANPRINT  goals.  This  approach  has  been 
attempted  in  the  past,  with  noticeably  disappointing  results.  A 
motivational  component  and  at  least  a  skeleton  means  of  linking  design 
principles  to  training  consequences  is  necessary,  in  addition  to  the 
design  principles  themselves. 

It  was  determined  that  a  composite  approach  could  satisfy  both  the 
motivational  and  linkage  requirements.  The  approach  chosen  is  to 
present  exemplar  training  problem  areas  (training  "high  drivers")  from 
the  histories  of  the  systems  and  subsystems  used  to  develop  the 
comparison  composite,  and  explain  their  impacts  on  training.  In 
addition,  we  will  present,  for  each  training  "high  driver,"  associated 
SSI  design  problems  which  have  been  the  causes  of  training  problems. 
Along  with  this  information,  SSI  design  principles  (or  references  to 
where  to  find  those  principles  and  how  to  apply  them)  will  be 
presented.  Tc  provide  context  and  present  a  complete  "training 
estimate"  package,  predicted  training  system  characteristics  will  also 
be  presented. 

These  data  perform  two  functions.  First,  they  sensitize  the 
designer  to  training  issues,  since  the  "most  probable"  training 
characteristics  estimates  are  presented.  Second,  they  promote 
(although  they  cannot  constrain  in  detail)  an  awareness  of  effective 
SSI  design  principles  and  the  impacts  of  poor  SSI  design  upon  training. 
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These  functions,  in  combination,  provide  the  SSI  designer  with  an 
opportunity,  and  hopefully  a  motive,  to  utiLize  effective  SSI  design 
principles  when  making  design  choices.  Utilizing  those  principles  will 
work  to  reduce  training  system  requirement* ,  and  meet  MANPRINT  training 
goals  for  the  new  system. 


Overview  of  the  Use  of  Product  Four 


Figure  1  illustrates  Che  general  nature  of  the  operation  of  the 
ASA/SAIC  Product  Four  concept.  The  following  subsection  presents  an 
overview  of  the  function  and  components  of  Product  Four  and  the  user 
interface  with  the  Product.  A  highly  detailed  scenario  of  the 
interactions  between  the  user  and  the  Product  and  the  functions 
performed  autonomously  by  the  Product  is  presented  in  Section  Three  of 
this  paper.  Section  Four  discusses,  in  detail,  the  Product's  data 
requirements . 

The  overall  function  of  Product  Four  is  divided  into  three  phases, 
preceded  by  Product  activation  and  followed  by  generation  of  final 
outputs.  The  Product  itself  contains  several  integral  databases  and 
two  ''engines"  for  performing  Product  functions. 


Act ivat ion 


When  Product  Four  is  activated,  the  user  selects  either  the 
operational  mode,  which  leads  to  Phase  I,  or  the  Embedded  Training  ( ET ) 
mode.  When  the  operational  mode  is  selected,  the  user  has  the  option 
of  entering  new  data  or  modifying  existing  input  data  that  exisc  in 
system  files. 


Phase  I  —  Development  of  Data  Entry  Template  and  Input  of  Data 

This  is  the  input  phase  of  Product  Four.  In  this  Phase,  relevant 
performance  characteristics  of  the  proposed  system  and  its  component 
subsystems  are  entered  into  a  data  input  template.  The  characteristics 
that  are  input  are  the  basis  for  identification  of  the  comparison 
system  against  which  training  estimates  will  be  developed.  The 
activities  in  Phase  I  are: 

1.  The  user  selects  the  class  (e.g.,  Close  Combat  -  Heavy)  and 
subclass  (e.g.,  Tank)  of  system  with  which  he  or  she 
desires  to  work,  based  on  prompting  provided  by  the 
Product . 

2.  User  selects  a  specific  or  default  template,  or  data  model, 
for  data  input  (provided  by  the  Product;  assistance  is  also 
provided  in  template  selection). 
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3.  User  customizes  the  selected  template  (with  Product  assist¬ 
ance,  as  required)  to  correspond  to  data  available  or  to 
specific  characteristics  of  the  proposed  system  for  which 
the  training  estimate  is  desired. 

4.  Following  the  data  model,  the  user  inputs  data  which 
describe  the  performance  characteristics  of  each  subsystem 
of  the  system  for  which  the  estimate  is  desired.  These 
data  are  the  basis  for  the  selection  of  the  comparison 
system. 

Phase  II  —  Comparison  Identification  and  Modification 

During  this  Phase,  the  Product's  composite  comparison  "engine” 
identifies  subsystems  from  the  resident  existing  systems  database  to 
form  a  (possibly)  composite  profile  of  subsystems  on  which  to  base  the 
training  characteristics  estimate.  When  the  identification  is 
complete,  the  user  has  an  opportunity  to  modify  the  input  data,  change 
the  subsystems  in  the  comparison  system  composite,  or  accept  the 
"engine's"  choices  without  modification.  There  are  two  components  of 
Phase  II  activities: 

1.  The  Product  exercises  the  composite  comparison  identi¬ 
fication  engine  and  selects  the  best-fit  comparison 
composite  for  training  characteristics  estimation;  alterna¬ 
tives  for  each  subsystem  (with  lower  figures  of  merit)  are 
also  identified. 

2.  The  user  reviews,  and  modifies  if  desired,  the  comparison 
composite  developed  by  the  Product.  The  user  may  override 
the  Product's  choices  in  any  fashion  deemed  necessary.  SME 
input  to  refine  the  composite  may  be  applied  at  this  point, 
if  available.  The  user  has  the  option  to  modify  input  data 
to  generate  other  comparison  composites. 


Phase  III  —  Training  and  Design  Characteristic  Generation  and 
Mod  if icat ion 


During  this  Phase,  the  Product's  training  and  design  character¬ 
istic  estimation  engine  selects,  from  two  resident  databases,  the 
training  system  characteristics,  design  and  training  problems,  and  SSI 
design  principles  data  which  are  the  output  of  Product  Four  as  a  whole. 
The  first  database  contains  training  system  characteristics  information 
on  each  system  and  subsystem  that  can  be  used  as  an  element  of  a 
composite  comparison  system.  The  second  database  contains  training 
"high  driver"  information,  design  problems  associated  with  training 
"high  driver"  factors,  and  references  to  SSI  design  principles  that  (if 
applied  effectively)  may  ameliorate  design  (and  training)  problems 
associated  with  the  "high  driver"  factors.  The  structure  and  contents 
of  these 
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databases  and  their  data  elements  are  discussed  in  Section  Four  of  this 
paper.  Phase  III  encompasses  the  following  activities: 


1.  The  Product's  training  characteristic  estimation  engine 
retrieves  and  conditions  data  for  the  training  character¬ 
istics  estimate,  and  makes  the  data  available  to  the  user. 

2.  The  user  reviews  the  output  of  the  training  characteristics 
estimate  generation  process,  and  modifies  training 
characteristics  as  desired,  using  any  available  or  mandated 
external  inputs. 

3.  The  Product  retrieves  training  "high-driver,"  design 
problems,  and  remediation  and  prescription  data  for  user 
review. 

4.  The  user  reviews  and  revises  the  data  as  required  or 
desired  to  customize  output  for  designers  or  system 
developers . 

Output 


The  user  can  select  any  of  the  generated  training  characterist  ics 
data  or  other  data  (or  combinations)  for  final  output.  The  user 
selects  the  desired  outputs  by  responding  to  Product  queries.  The 
selected  information  is  then  formatted  and  output  for  ultimate  use  in 
the  acquisition  process. 

The  user  is  supplied  with  extensive  helps,  defaults,  data  models, 
and  embedded  training  availability  throughout  Product  Four  use.  These 
Product  features  make  it  possible  to  generate  at  least  initial 
"stravman"  training  characteristics  with  minimal  data  or  expertise  in 
computer  operation.  In  addition,  the  user  is  completely  free  to 
re-enter  the  Product  operation  cycle  at  any  point,  to  modify  input  or 
composite  comparison  data,  or  to  alter  output  data  based  on  factors  not 
applied  in  the  previous  operation  of  the  Product. 


Overview  of  the  Remainder  of  This  Paper 


The  balance  of  this  paper  consists  of  eight  sections,  as 
follows : 


Section  Two  details  the  information  that  Product  Four  will 
produce  as  a  result  of  its  operation,  and  the  aiding  that 
will  be  provided  for  the  operational  user  of  the  Product. 
This  corresponds  to  requirement  number  four  for  the  concept 
paper  in  the  contract  Statement  of  Work  (SOW). 
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Section  Three  presents  a  detailed  discussion  of  how  Product 
Four  will  function  to  produce  its  outputs,  as  a  detailed 
scenario  including  descriptions  of  user  interactions  with 
the  Product.  This  corresponds  to  SOW  requirement  five. 

Section  Four  discusses  specific  data  requirements  for 
Product  Four,  and  details  the  sources  from  which  data  will 
be  accessed  or  derived,  as  well  as  procedures  for 
developing  data.  This  corresponds  to  SOW  requirements  two 
and  three. 

Section  Five  discusses  the  time  required  to  use  Che  Product 
Four  system  to  develop  training  characteristics  estimates. 
This  corresponds  to  SOW  requirement  nine. 

Section  Six  describes  how  Product  Four  will  be  developed. 
This  corresponds  to  SOW  requirement  one. 

Section  Seven  describes  the  means  that  will  be  built  into 
Product  Four  to  train  its  users  and  facilitate  their  use  of 
the  Product.  This  corresponds  to  SOW  requirement  6even. 

Section  Eight  discusses  our  approach  to  gaining  acceptance 
and  use  of  the  outputs  of  Product  Four.  This  corresponds 
to  SOW  requirement  six. 

Section  Nine  discusses  our  approach  to  attaining 
institutionalization  of  Product  Four  in  the  Army  context  as 
it  presently  exists.  This  corresponds  to  SOW  requirement 
ten. 
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SECTION  TWO 


INFORMATION  AND  AIDING  WHICH  PRODUCT  FOUR  PROVIDES 


Product  Four  provides  information  and  aiding  to  three  general 
classes  of  personnel.  These  are: 

1.  The  actual  users  of  Product  Four  -  the  analyst  with 
responsibility  for  estimation  of  training  characteristics 
for  MANPRINT  ("the  user"). 

2.  The  general  system  and  SSI  designers  ("the  designers")  who 
utilize  the  outputs  of  Product  Four  as  targets  and 
constraints  in  the  process  of  system  and  SSI  design. 

3.  System  developers  and  managers  ("the  developers")  who  use 
the  estimates  provided  by  Product  Four  as  reference  and 
evaluation  information  in  assessing  whether  system  designs 
meet  MANPRINT  training  goals. 

The  characteristics  of  information  and  aiding  provided  to  each 
class  of  personnel  is  somewhat  different.  Specifics  are  discussed  in 
the  following  paragraphs. 


The  User 


Since  this  Product  represents  an  attempt  to  automate  a  complex 
process  and  enable  the  user  (who  is  not  expected  to  be  trained  in  the 
nuances  of  the  training  estimation  process)  to  make  training 
characteristics  estimates,  the  most  detailed  assistance  is  provided  to 
the  user.  Product  Four  will  provide  the  following  general  classes  of 
aiding  to  the  user. 

1.  Assistance  in  Developing  Input  Data.  Product  Four  will 
provide  both  default  and  customized  templates  or  data 
models  for  structuring  input  data  that  describe  the 
characteristics  of  the  new  system  for  which  a  training 
estimate  is  required.  Standardized  templates  will  be 
provided  for  each  class  and  subclass  of  system  for  which 
the  Product  will  support  estimate  development.  The 
templates  will  include  identification  of  subsystems, 
comparability  dimensions,  tolerances,  and  comparison  rules. 
In  all  cases,  defaults  will  be  available  to  enable  the  user 
to  work  with  preliminary  or  incomplete  data  and  still  be 
able  to  derive  a  "strawman"  or  first-approximation 
estimate.  Input  data  developed  in  each  instance  of  °roduct 
use  will  be  retained  on  file  for  future  use  as  a  template 
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candidate,  or  for  modification  for  updates  or  sensitivity 
analyses  of  original  estimates.  An  example  of  one  portion 
of  a  data  model  is  shown  in  Figure  2. 

2 .  Override  and  Tailoring  Opportunity  for  Comparison  Cases. 

In  some  cases,  the  user  may  have  direct  access  to  SME 
resources  who  can  critique  and  suggest  modifications  to  i.ne 
comparison  cases  identified  by  Product  Four.  Also,  the 
user  may  occasionally  be  sufficiently  sophisticated  or 
knowledgeable,  or  presented  with  external  constraints,  to 
modify  the  suggested  composite  cases  developed  by  the 
Product  on-line  or  autonomously.  An  example  of  an  external 
constraint  is  "cen  percent  less  training  shall  be  required 
overall  than  for  Che  immediate  predecessor  system."  Each 
comparison  case  developed  by  the  system  is  subject  to 
review  and  modification  by  the  user  at  any  time,  even 
during  intermediate  stages  of  estimate  development.  Each 
estimate  generated  by  the  Product  or  modified  by  the  user 
is  retained  on  file  for  later  retrieval,  modification,  and 
change.  This  supports  a  high  degree  of  capability  for 
sensitivity  analyses  and  introducing  additional  factors 
from  external  sources  (e.g.,  SMEs ,  if  available)  to  support 
training  charac ter ist ics  estimation. 

3 .  Override  and  Tailoring  Opportunity  for  Product  Outputs. 

The  automation  inherent  in  Product  Four  cannot  guarantee 
outputs  that  will  be  completely  satisfactory  for  all 
possible  purposes.  For  example,  a  training  system  estimate 
may  exceed  parameters  that  have  already  been  established 
for  the  new  system.  Thus,  the  user  needs  an  opportunity  to 
review,  modify,  and  tailor  the  training  system  estimate  to 
meet  a  variety  of  conditions.  Likewise,  the  user  may  have 
insight  into  the  amount  and  level  of  training  "high  driver" 
information,  design  impacts  on  training,  and 
remediation  and  prescription  information  that  is  acceptable 
and  usable.  This  information  can  also  be  reviewed  in 
detail,  and  the  Product-suggested  outputs  tailored  so  that 
the  unique  needs  of  Product  output  users  can  be  served. 

4.  Sensitivity  Analysis  Capability.  In  many  cases,  alternate 
impacts  of  different  assumptions,  different  input  data,  or 
different  composite  comparison  systems  upon  training  system 
estimates  may  need  to  be  generated.  Product  Four  enables 
the  user  to  suspend  Product  operation  in  order  to  modify 
input  or  intermediate  Product  data  at  many  points.  Using 
this  capability,  the  user  can  explore  both  the  impacts  of: 
(a)  varying  input  data  on  the  comparison  systems  identified 
by  the  product;  and  (b)  varying  input  or  comparison 
composite  case  data  on  the  ultimate  outputs  of  Product 
Four.  The  product  also  supports  serial  iteration  through 
the  entire  Product  function  cycle  for  sensitivity  or  impact 
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DATA  INPUT  TEMPLATE 


For  New  System:  AHX  (Attack  Helicopter) 


Inputing  Data 

for  Subsystem: 

Commun icat ions 

Characteristic 

Template 

Value 

New-System 

Value 

UHF  Radio  Power 

UHF  Radio  #  Channels 

200  watts 

720 

200  watts 

720 

VHF  Radio  Power 

VHF  Radio  #  Channels 

145  watts 

320 

145  watts 

640 

HF  Radio  Power 

HF  Radio  #  Channels 

100  watts 

240 

125  watts 

240 

Crypto  Capability  (Sys) 

Yes 

o 

New  Data  Input 

Filename:  AHX1NEW. V01 

Options:  NEXT-SUB  DONE 

PREV-SUB 

HELP  ABORT 

CHK-PNT 

SAVE-END  PRINT 

Enter  Value  at  Cursor 


Figure  2.  Data  input  template. 
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analyses.  This  is  done  by  preserving  all  input  data  or 
comparison  case  alternatives  investigated.  This  enables 
the  user  to  maintain  an  audit  trail  of  sensitivity  analyses 
conducted  and  make  input  and  output  data  available  as 
possible  templates  and  data  models  for  future  product  use. 

5.  Embedded  Training  and  On-line  Helps.  Product  Four  will 
have  a  comprehensive  embedded  training  component  which  will 
have  the  capability  to  instruct  the  operator  in  both  basic 
and  advanced  use  of  the  Product,  as  well  as  the  underlying 
methods  embedded  in  the  way  the  Product  functions. 

Training  capabilities  will  also  cover  the  why  and  how  of 
conducting  sensitivity  .analyses.  A  context-sensitive 
on-line  help  facility,  coupled  to  the  embedded  training 
capability,  will  also  be  provided  by  Product  Four.  Help 
will  always  be  available  to  both  the  novice  and  experienced 
user  to  enable  the  users  to  accomplish  their  goals  in  using 
the  Product. 

6.  Override  of  Product  Steps.  This  option  is  available  for 
those  experienced  users  who  prefer  to  override  steps  in 
Product  Four  function  for  direct  entry  or  for  other 
reasons.  This  option  is  available  at  most  steps  in  Product 
Four  function,  as  indicated  in  the  detailed  functional 
description.  Override  is  accomplished  through  commands 
given  when  the  Product  is  activated. 

7.  Processing  Flexibility.  Batch  processing  and  accelerated 
processing  are  available  for  more  sophisticated  users,  or 
for  those  desiring  to  conduct  exploratory  or  follow-up 
sensitivity  analyses  or  validate  databases  and  product 
operation. 


The  Designer 


As  discussed  in  the  philosophy  of  Product  Four  in  Section  One  of 
this  paper,  one  objective  of  the  Product  is  to  provide  information  that 
assists  the  designer  to  act  in  such  a  fashion  as  to  meet  MANPRINT 
training  goals.  To  reiterate,  the  information  that  will  be  provided 
for  designers  is  as  follows: 

1.  Training  System  Characteristics  Estimate.  An  estimate  of 
the  training  time  for  all  classes  of  operator  and 
raaintainer  personnel  for  which  data  are  available,  by 
subsystem  and  for  the  system  at  large.  The  training  system 
characteristics  estimate  will  also  provide  an  estimated 
training  device  requirements  profile  (type  and  number  of 
devices)  for  training  each  class  of  operator  and 
maintenance  personnel. 
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2.  Training  "High  Drivers"  Data.  The  Product  will  provide  the 
system  and  SSI  designer  with  critical  incident  descriptions 
of  cases  in  the  histories  of  the  subsystems  which  make  up 
the  comparison  case  that  represent  excessive  training 
requirements.  The  "high  drivers"  information  will  be 
related  to  cases  where  system  (or,  more  specifically,  SSI) 
design  is  attributed  as  a  principal  factor  in  the  existence 
of  training  high  driver  factors. 

3.  Design  Problems  Related  to  High  Drivers.  Product  Four  will 
provide,  for  each  "high  driver"  example  output  to  users,  a 
diagnosis  of  design  problems  which  have  been  implicated  as 
causal  factors  in  creating  unnecessary  or  excessive 
training.  These  examples  will  be  as  specific  as  possible, 
to  allow  designers  to  derive  object  lessons  of  the  impacts 
of  poor  SSI  or  system  design  upon  training. 

4.  Remediation  and  Prescription  Data.  The  final  elements  of 
information  output  for  designers  will  be  reference  to  or 
citation  of  human  engineering  design  principles  which,  if 
applied,  might  have  ameliorated  the  design  problems 
associated  with  training  "high  driver"  factors.  The 
purpose  of  presenting  this  reference  data  is  to  influence 
design  choices  toward  design  characteristics  that  minimize 
training  needs  for  the  system. 


The  System  Developers 


Acquisition  and  system  development  personnel  have  uses  for  the 
training  system  characteristics  information  produced  by  Product  Four. 
This  information  may  be  useful  as  (for  example)  feeder  data  for 
developing  Organizational  and  Operational  (0&0)  plans  for  the  new 
system.  Also,  if  the  training  system  characteristics  data  produced  by 
Product  Four  are  accepted,  they  may  be  the  basis  for  developing 
MANPRINT  criteria  and  constraints  in  acquisition  documents,  and  as 
reference  values  for  evaluating  system  design  compliance  with  MANPRINT 
constraints . 
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SECTION  THREE 


PRODUCT  FUNCTION 


General  Features  of  Product  Four  Function 


The  operational  structure  of  Product  Four  is  illustrated  in  Figure 
3.  Product  Four  is  functionally  divided  into  three  Phases  that  are 
described  in  this  Section,  preceded  by  Product  activation  and  followed 
by  output  of  data.  There  are  certain  characteristics  (presented  in  the 
previous  section  of  this  concept  paper)  underlying  all  of  the  phases  of 
Product  Four  function  that  add  to  its  ease  of  application  (user 
friendliness).  These  characteristics  include  aspects  of  the  user 
interface  and  of  the  Product's  resident  databases. 

The  data  requirements  as  well  as  the  user  interface  requirements 
of  Product  Four  are  based  on  the  overriding  goal  to  make  the  Product  as 
user  friendly  as  possible.  The  types  of  data  needed  to  develop  and 
exercise  Product  Four  are  illustrated  in  Figure  3.  To  summarize,  they 
are:  (1)  input  data,  (2)  comparison  systems  data,  (3)  training  and 
design  charac terist ics  data,  and  (4)  remediation  and  prescription  data. 
A  basic  assumption  is  that  the  user  of  Product  Four  will  have  little 
time  or  opportunity  to  find  and  consult  with  SMEs  in  order  to  apply 
Product  Four.  Therefore,  Product  Four  will  incorporate  subject  matter 
expertise  within  its  resident  databases  to  ensure  the  at  least  a  "best 
guess"  is  delivered  to  the  user.  Confidence  in  Product  Four  outputs 
will  be  a  function  of  the  completeness  and  accuracy  of  the  input  data, 
background  knowledge  of  the  user  (as  reflected  by  the  number  of 
defaults  used)  and  completeness  and  accuracy  of  the  resident  databases. 
Data  included  in  the  Product  Four  resident  databases  will  ensure  that 
the  user  will  always  have  a  "strawraan"  output  which  can  be  refined  as 
the  user  sees  fit.  Therefore,  defaults  will  be  available  for  every 
user  decision  required  except  the  type  of  system  for  which  a  training 
characteristics  estimate  is  desired. 

The  remainder  of  this  Section  presents  a  detailed  description  of 
Product  Four  function,  supplemented  with  flowcharts  that  illustrate  the 
functioning  of  the  Product. 

NOTE:  In  the  discussion  below,  functions  that  are  marked  with  an 

asterisk  (*)  are  able  to  be  specified  by  parameter  entries  when  Product 
Four  is  activated.  When  this  is  done,  the  detailed,  menu-driven 
interactions  associated  with  inputs  that  are  specified  during 
activation  do  not  take  place.  This  is  a  feature  included  for 
experienced  users,  to  facilitate  rapid  generation  of  estiraat***  in  (for 
example)  conducting  sensitivity  analysis. 
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MANPRINT  PRODUCT  4 

CONCEPT  OE  OPERATIONAL  STRUCTURE 


Product  Four  Activation 


The  activation  of  Product  Four  is  illustrated  by  the  flowchart  in 
Figure  4.  Upon  activation,  the  Product  Four  user  Belects  either  the  ET 
or  operational  mode.  The  ET  capability  is  described  in  detail  in 
Section  Seven.  The  operational  mode  takes  the  user  directly  into 
Product  functions.  The  user  can  exit  the  operational  mode  at  any  point 
and  return  to  Chat  point  at  a  later  time.  Once  in  the  operational 
mode,  the  user  indicates  whether  or  not  a  new  estimate  is  to  be 
derived.  When  a  new  estimate  will  be  developed,  the  Product 
automatically  takes  the  user  to  Phase  I.  When  the  user  desires  to 
modify  existing  data  or  to  continue  work  on  an  estimate  already  begun, 
he  or  she  selects  the  file  to  be  retrieved.  Product  Four  then  begins 
func  t ioning . 


Phase  I  —  Development  of  Data  Entry  Template  and  Input  of  Data 


Overv iew 


This  phase  encompasses  data  model  (template)  development  and  data 
input.  The  flowcharts  in  Figure  5  illustrate  Phase  I  functions. 

During  data  model  development,  the  analyst  enters  data  associated  with 
relevant  performance  characteristics  (dimensions)  of  the  proposed 
system  and  its  component  subsystems.  The  principal  reason  for  using 
templates,  or  data  models,  to  guide  input  is  to  ensure  that  the  input 
data  conform  to  the  types  of  data  and  units  of  measure  which  are 
contained  in  the  comparison  systems  database.  When  the  data  model  has 
been  completed,  data  that  describe  the  characteristics  of  the  proposed 
system  and  its  subsystems  are  entered  in  the  format  of  tne  template. 
These  data  are  the  basis  for  identifying  the  comparison  case  in  Phase 
II. 


Outcome 


At  the  end  of  this  phase,  the  data  model  is  completed,  and  the 
pcrfcrmcr.ce  characteristics  and  associated  values  of  the  component 
subsystems  of  the  proposed  system  have  been  entered. 


Detailed  Functional  Description 

Step  1.1  Identification  of  the  class  of  system  for  which  the  estimate 
is  desired. 
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User  Activates 
Protect  Four 


(Protect  yuories/ 
whether  ui*r 
(wishes  training 
or  operation 
of  Protect 


NOT!  A:  User  ear 
ovtmdt  this  stop  Ay 
incluiinf  commas 
option  on  invocation 
of  software  fron 
opera  tins  syste* 
level  (Amin  TO 

several  sunnuixi 

STEPS) 


Product  Four  on  tors 
E*Aete*4  Tminjny 
cowpenent;  provisos 
train  my  as 
reyuired  Ay  «sor  — 
off lino 


Protest  yueries 
usor  whether 
an  ontiroly  now 
•stiwato  is 
tesire4,  or 
roviow/  upteto 
of  an  existiny 
•stiwato  is 
fisirof 


Protest  ouorios 
usor  for  filonaao 
an  4  vorsion  of 
location  of  input 
teta  to  Ao  refined 
(NOTE  A  Applios) 


Usor  ontors 
file  and 
vorsion  teta; 
protest 
retrieves 

*t tr1*'* 


Figure  4.  Product  activation  flowchart. 
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Bey  in  PHAJI  0 NX 


Tier  1.1 


Product  yueries 
user  for  class  of 
systen  for  uhiek 
estinate  if  4esire4 
(Arnor,  Aviation, 
Artillery,  etc.) 
[HOT I  A  Applies] 


User  enters 

/class  of 
listen  (fron 
nenu) 


L_ 


<»>  u 


If  there  are 
various  subclasses 
if  systsM  in 
yeneral  class, 
pro4uct  fueries 
user  input  of 
4*sire4  sum  lass  ta 
use  as  4ata  no4tl 
(NOT!  n  Applies! 


NOTE  »:  User  is 
presente4  uith  nenu 
an4  noviny  cursor 
uitk  bypass 
capability  an4 
cewun4  entry  option 
to  oeerri4e  nenu  if 
4esire4  (APPLID  TO 
LATH  JUKI 


User  selects 
subclass  or 
4*fault.  as 
4esire4 


Jter  1.3 


If  there  are  nultiple 
systens  in  subclass, 
pro4uct  yueries  user 
as  to  ohether  a 
specific  systen's 
4ata  are  4esire4  as 
an  input  4ata 
tenplate  [NOTEJ  A  an4 
I  Apply! 


User  enter* 
systen  none, 
or  selects 
4efault 


Jter  1.4 


Pre4uct  retrieves 
4a ta  ns 4*1/ 
tenplate  for 
specific  systen 
(user  choice)  or 
yeneric  tenplate 
for  systen  class  t 
subclass 


»tf  1,:?. 


Pro4uct  4i splays  a 
nenu  of  subsysten* 
for  selecte4  or 
yeneric  tenplate 
an4  yueries  uhether 
user  uishes  to 
4isreyar4  any 
subsystew  (NOTH  A 
an4  II 


Pro4uct  4isplays 
inventory  of  systens, 
an4  yueries  uhether 
user  uishes  to 
exclu4e  any  systens 
as  possible 
conparison  can4i4ates 

[NOTH  A  an4  I  Apply) 


User  selects  any 
systens  4esir*4 
to  be  exclu4*4 
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A  an4  I  Apply] 


Figure  5.  Phase  I  flowchart. 
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Figure  5.  Phase  1  flowchart  (continued). 
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Rationale:  The  identification  of  the  proposed  system  class  is 
necessary  for  two  purposes.  First,  because  selecting  a  data  model 
template  for  data  entry  is  made  easier  if  the  system  class  is 
specified,  and  second,  because  data  in  the  comparison  systems 
database  is  organized  by  system  class. 

♦Product  Action:  Presents  the  user  with  a  menu  of  classes  of 
systems  (e.g. ,  Aviation,  Armor,  Field  Artillery,  etc.). 

User  Action:  Selects  Che  class  of  system  for  which  the  estimate 
is  required. 

Help  Available:  In  most  cases  the  system  class  will  be  obvious  to 
the  analyst.  For  chose  rare  occasions  when  the  class  of  the 
proposed  system  is  not  immediately  apparent ,  prompts  and  cues  will 
be  provided,  including  class  definitions  and  examples  of  systems 
in  each  class,  to  aid  in  the  identif icat ion  of  the  appropriate 
class. 

Default:  A  default  option  will  not  be  available;  the 
identification  of  a  class  of  system  for  data  model  development  is 
requ ired . 

Step  1.2  Ident if icat ion  of  the  subclass  of  the  proposed  system. 

Rationale:  Once  again,  the  identification  of  the  subclass  narrows 

down  the  possibilities  for  the  data  entry  template  and  potential 
comparison  systems. 

♦Product  Action:  Presents  the  user  with  subclasses  of  systems  in 
the  selected  class  (e.g.,  Attack  Helicopters,  Cargo  Helicopters, 
Utility  Helicopters,  Fixed  Wing  Aircraft,  etc.  within  the  Aviation 
class). 

User  Action:  Selects  the  subclass  required. 

Help  Available:  In  most  cases,  the  system  subclass  will  be 
obvious  to  the  user.  When  the  user  cannot  determine  whether  the 
proposed  system  falls  into  one  of  the  presented  subclasses,  the 
Product  can  provide  decision  aids  (through  the  Help  facility) 
including  descriptions  of  systems  included  in  each  subclass. 

Default:  A  default  option  will  not  be  available;  the 

identification  of  a  subclass  of  system  for  data  model  development 
is  required. 

Step  1.3  Selection  of  initial  data  input  template. 

Rationale:  Data  input  must  conform  to  a  data  model  that  reflects 

the  types  of  data  and  units  of  measure  available  in  the  comparison 
systems  database  for  the  particular  class  or  subclass  of  system 
for  which  the  estimate  is  desired. 
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♦Product  Action:  Presents  the  user  with  a  list  of  systems  in  the 
subc  lass . 

User  Action:  Selects  a  system  whose  data  will  be  used  as  an  input 
data  template,  or  chooses  to  use  a  generic  template  for  data 
input.  The  next  step  is  not  performed  if  the  user  chooses  to  use 
a  generic  data  model,  rather  than  one  based  on  a  specific  system, 
on  the  assumption  that  a  user  who  requests  a  generic  data 
model  will  not  be  sufficiently  sophisticated  to  be  able  to  exclude 
specific  systems  from  comparison  candidacy. 

Help  Available:  Lists  of  systems  and  their  component  subsystems 
in  the  comparison  systems  database  will  be  available.  In 
addition,  the  user  can  review  templates/data  models  associated 
with  the  systems  in  the  comparison  systems  database  as  required, 
to  aid  in  selection. 

Default:  Data  input  will  be  based  upon  a  generic  data 
model  corresponding  to  the  subclass  of  the  proposed  system. 

Step  1.4  Elimination  of  systems  as  comparison  candidates. 

Rationale:  The  elimination  of  systems  as  potential  comparison 

candidates  allows  the  user  to  directly  impact  the  nature  of  the 
comparison  to  suit  the  individual  estimate  under  development. 

♦Produc :  Action:  Presents  the  user  with  a  list  of  systems  in  the 
selected  subclass. 

User  Action:  Selects  systems  to  be  eliminated  as  possible 
composite  comparison  candidates  (if  any). 

Help  Available:  None.  Users  who  wish  to  exclude  specific  systems 
as  potential  comparison  candidates  are  presumed  to  have  reasons 
for  doing  so,  or  to  have  been  directed  to  do  so. 

Default:  All  of  the  systems  in  the  identified  class  or  subclass 

will  be  considered  potential  composite  comparison  candidates. 

Step  1.5  Elimination  of  subsystems  in  the  data  model  that  are  not 
applicable  to  the  proposed  system. 

Rationale:  There  may  be  times  when  the  proposed  system  does  not 

include  all  of  the  subsystems  in  the  selected  template;  the 
elimination  of  inapplicable  subsystems  is  the  initial  tailoring  of 
the  input  template  so  that  it  will  eventually  reflect 
characteristics  of  the  proposed  system. 

♦Product  Action:  Presents  the  user  with  a  list  of  subsystems  of 
the  system  chosen  for  the  data  model. 
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User  Action:  Indicates  subsystems  to  exclude  from  the  data  model 
( 1 f  any ) . 

Help  Available:  None.  Since  subsystems  are  only  eliminated  when 
they  are  not  part  of  the  proposed  system,  it  is  expected  that  this 
step  will  be  easily  accomplished  without  additional  help.  Users 
must  have  available  the  names  of  subsystems  as  part  of  input 
data. 

Default:  All  of  the  subsystems  in  the  selected  template  will  be 
included  in  the  data  model,  and  are  assumed  to  be  present  in  the 
proposed  system. 

Step  1.6  Addition  of  subsystem(s)  to  template. 

Rationale:  The  selected  input  template  may  not  include  all  of  the 

subsystems  in  the  proposed  system.  The  user  has  the  option  of 
adding  new  subsystems  to  the  data  entry  template. 

♦Product  Action:  Presents  user  with  a  list  of  the  subsystems 
included  in  the  data  template  and  the  option  of  adding  a 
subsystem. 

User  Action:  User  decides  whether  a  subsystem  should  be  added  to 
the  template.  When  a  subsystem  is  to  be  added,  the  user  enters 
the  subsystem  name.  If  no  subsystems  are  to  be  added,  the  next 
step  is  not  performed. 

♦Product  Action  (following  user  action):  If  a  subsystem  is  to  be 
added  to  the  template,  the  Product  retrieves  and  displays  the 
names  of  all  of  the  systems  in  the  comparison  systems  database 
which  have  subsystems  named  the  same  as  the  subsystem  to  be 
added  . 

Help  Available:  The  user  may  request  a  listing  of  unique 
subsystem  names  (from  the  comparison  systems  database)  and  display 
the  list  to  the  user  as  an  aid  in  specifying  a  name  for  the 
subsystem  to  be  added. 

Default:  Additional  subsystems  will  not  be  added  to  the 

template . 

Step  1.7  Identification  of  comparison  dimensions,  values,  and 

tolerances  for  the  template  of  the  added  subsystem! s ) . 

Rationale:  Comparison  dimensions  and  values  must  be  specified  to 
allow  comparison  of  input  data  with  data  in  the  comparison  systems 
database.  This  step  enables  the  user  to  include  or  customize 
dimensions  and  values  in  the  input  template  that  reflect  the 
characteristics  of  the  subsystem  to  be  added. 
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♦Product  Action:  Presents  a  list  of  systems  containing  subsystems 
with  the  same  name  as  the  added  subsystem,  along  with 
corresponding  comparison  dimensions  and  values  (values  identify 
the  units  of  measure). 

User  Action:  Reviews  and  selects  from  the  comparison  dimensions, 
values,  and  tolerances  associated  with  applicable  subsystems, 
selects  one  system  whose  data  will  provide  the  template,  or  enters 
this  information  independently. 

Help  Available:  Will  include  prompts  and  cues  for  entering  and 
selecting  dimensions,  values,  and  tolerances. 

Default*  A  generic  template  for  the  added  subsystem(s)  will  be 
utilized . 

Step  1.8  Entry  of  input  data  according  to  the  template,  by  subsystem. 

Rationale:  The  template  that  exists  to  this  pc^nt  serves  as  the 

model  for  data  input  for  the  system  for  which  the  estimate  is 
required.  During  this  step,  data  is  input  to  reflect  the  expected 
characteristics  and  the  comparison  priorities  of  the  proposed 
system. 

♦Product  Action:  Presents  the  user  with  a  list  of  subsystems 
currently  in  the  template. 

User  Action:  User  selects  a  subsystem  for  modification. 

♦Product  Action:  Presents  the  user  with  a  list  of  the  selected 
subsystem's  characteristics,  along  with  the  associated  units  of 
measure.  Figure  6  shows  an  example  of  a  conceptual  input  screen 
for  a  subsystem. 

User  Action:  User  enters  specific  characteristics  data  according 
to  the  template.  User  may  also  assign  comparison  priorities  to 
each  characteristic,  or  allow  them  to  be  equally  weighted. 

NOTE:  The  four  steps  immediately  above  are  iterated  until  the 
user  has  completed  entering  input  data  according  to  the  data 
model . 

Help  Available:  Extensive  help  messages  will  be  available 
including:  pertinent  definitions;  reviews  of  the  use  of  the 

information  to  be  entered  during  comparison  case  development; 
considerations  for  making  priority  decisions;  and  other  (to  be 
determined)  prompts  and  decision  aids. 

Default:  The  existing  data  from  the  comparison  systems  database 

for  the  subsystem  under  consideration  will  be  used  as  is  (i.e., 
without  user  modifications)  as  input  data  for  the  subsystem  of  the 
new  system. 
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DATA  INPUT  TEMPLATE 


For  New  System:  AHX  (Attack  Helicopter) 


Inputing 

Data 

for  Subsystem: 

Commun ic at  ions 

Characteristic 

Template 

Value 

New-System 

Value 

UHF  Radio  Power 

UHF  Radio  #  Channels 

200  warts 

720 

200  watts 

720 

VHF  Radio  Power 

VHF  Radio  #  Channels 

145  watts 

320 

145  watts 

640 

HF  Radio  Power 

HF  Radio  #  Channels 

100  watts 

240 

125  watts 

240 

Crypto  Capability  (Sys 

) 

Yes 

<~> 

New  Data  Input 

Filename:  AHX1NEW. V01 

Options:  NEXT-SUB 

PREV-SUB 

DONE 

HELP  ABORT 

CHK-PNT 

SAVE-END  PRINT 

Enter  Value  at  Cursor 


Figure  6.  Data  input  template. 


El-30 


NOTE:  Using  default  values  from  the  comparison  systems  database 

is  one  means  by  which  the  user  can  deal  with  specific  external 
constraints  on  training  characteristics.  For  example,  a  directive 
may  have  been  issued  that  specifies  that  training  time  for  the  new 
system  will  be  some  percentage  less  than  that  for  a  particular 
predecessor  system.  In  such  a  case,  the  user  can  simply  default 
to  the  predecessor  system  (by  using  all  of  the  characteristics  and 
values  for  the  predecessor  system),  and  adjust  the  training 
characteristics  estimate  generated  by  Product  Four  to  conform  to 
the  external  constraints. 

Likewise,  if  some  knowledge  exists  that  optimum  training  (whatever 
that  may  be)  will  be  provided  for  a  contemplated  new  system,  the 
user  can  make  a  similar  sort  of  adjustment.  Although  it  is 
difficult  to  imagine  a  case  where  this  would  be  done,  Product  Four 
and  the  user  are  able  to  deal  with  such  a  situation.  In  this 
case,  the  user  might  conduct  a  sensitivity  analysis  to  determine 
the  maximum  training  time  and  resources  estimates  that  can  be 
generated  for  a  particular  kind  of  system,  and  use  those  values  as 
an  initial  estimate. 

Step  1.9  Data  management. 

Rationale:  The  input  data  must  be  stored  prior  to  retrieval  for 

use  during  the  composite  comparison  identification.  This  provides 
an  audit  trail  and  maintains  the  input  data  available  for  possible 
later  modification  or  re-use. 

User  Action:  Indicates  when  the  input  data  for  all  subsystems  is 
complete. 

Product  Action:  Detects  when  user  has  finished  inputing  data,  and 
creates  a  disk  file  that  includes  the  input  data  in  the  format  of 
the  data  model. 

Help  Available:  None.  Data  management  is  automatic. 

Default:  Data  management  is  automatic. 


Phase  II  —  Comparison  Case  Identification  and  Modification 


Overview 

The  flowchart  contained  in  Figure  7  illustrates  Phase  II  function. 
After  the  data  describing  the  characteristics  of  the  proposed  system 
are  entered,  the  composite  comparison  engine  compares  the  input  data, 
subsystem  by  subsystem,  with  data  in  the  comparison  systems  database. 
All  of  the  subsystems  corresponding  to  the  class  or  subclass  of  the 
input  system  are  considered  potential  comparison  candidates  (with  the 
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Step  2.1  -  Operation  of 
the  Comparison  Case 
Identification  Engine 


Figure  7.  Phase  II  flowchart. 
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exception  of  those  explicitly  excluded  in  Phase  I).  The  comparison 
identification  engine  selects  the  subsystems  that  are  the  best  matches 
to  the  input  data  through  a  template  matching  process.  Each 
characteristic  of  each  subsystem  in  the  input  data  is  systematically 
compared  to  the  applicable  subsystems  in  the  comparisons  systems 
database.  The  template  matching  is  performed  according  to  the 
comparison  rules  and  tolerances  contained  in  the  comparison  systems 
database  and  the  comparison  priorities  provided  by  the  user. 

The  template  matching  process  results  in  selection  of  the 
subsystems  which  most  closely  approximate  the  performance 
characteristics  included  in  the  input  data.  The  comparison  case 
identification  engine  computes  a  figure  of  merit  or  measure  of 
similarity  between  the  input  data  (on  a  subsystem  basis)  and  each 
like-type  subsystem  in  the  comparison  systems  database.  The  comparison 
subsystems  are  each  then  sorted  into  order  based  on  the  figures  of 
merit  computed.  In  aggregate,  the  subsystems  with  the  highest  figures 
of  merit  make  up  the  recommended  comparison  case  from  which  the 
training  system  character ist ics  estimate  is  derived. 

The  comparison  case  can  thus  be  a  composite  of  subsystems  from  as 
many  existing  systems  as  needed  to  fall  within  the  specified 
performance  envelopes  of  the  proposed  system.  Worst-case  estimates 
will  be  used  when  no  subsystems  in  the  comparison  systems  database 
match  the  input  data,  or  when  performance  envelopes  cannot  be  satisfied 
at  all . 

After  the  comparison  case  identification  engine  has  been 
exercised,  the  information  generated  is  presented  to  the  user  for 
review.  At  this  point,  the  user  reviews  the  subsystems  selected  for 
inclusion  in  the  comparison  case  by  the  comparison  case  identification 
engine.  The  user  can  exercise  several  options  following  this  review. 
First,  input  data  can  be  modified  and  a  new  comparison  case  generated 
to  reflect  changes  resulting  from  the  revised  input.  Second,  the  user 
can  accept  the  comparison  case  generated  by  Product  Four.  Finally,  the 
user  may  modify  the  comparison  case  by  substituting  different  subsystem 
choices,  after  reviewing  possible  choices  with  lower  figures  of  merit. 


Detailed  Functional  Description 

NOTE:  Step  numbers  in  the  discussion  below  refer  to  numbered 

steps  in  the  Figure  7  flowchart. 

Step  2.1  Operation  of  the  comparison  case  identification  engine. 

Rationale:  The  comparison  case  identification  engine  is 
operated  to  compare  each  of  the  subsystems  in  the  input 
data  with  every  applicable  comparison  subsystem  in  the 
comparison  systems  database. 

User:  The  user's  only  responsibility  at  this  point  is  to  indicate 
that  the  comparison  case  identification  engine  should  be 
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activated.  This  is  normally  done  by  indicating  that  input  of  data 
is  complete. 

Product  Action:  Product  Four  compares  the  comparison  attributes 
in  the  input  data  with  those  in  the  comparison  systems  database, 
in  the  following  manner: 

Substep  1.  Product  retrieves  the  input  data  for  all  attributes  of 
input  subsystem  J  (all  comparison  attributes  and 
associated  values). 

Substep  2.  Product  retrieves  comparison  attributes,  values,  and 
comparison  rules  for  the  Kth  subsystem  example 
comparable  to  input  subsystem  J,  from  the  comparison 
systems  database. 

Substep  3.  Product  compares  each  comparison  attribute  in  the 

input  data  with  each  comparable  comparison  attribute 
from  the  comparison  systems  database,  according  to  the 
rule  for  the  attribute.  If  the  input  data  fall  within 
the  strictest  bounds  established  by  the  comparison 
rule,  a  "1"  is  scored  for  the  attribute.  If  the  input 
data  fall  within  relaxed  bounds  established  by  the 
comparison  rule,  a  ".5"  is  scored  for  the  attribute. 

If  the  input  data  fall  outside  of  bounds  established 
by  the  comparison  rule,  a  "0"  is  scored  for  the 
attribute . 

Substep  4.  The  comparison  process  is  iterated  across  all 

comparison  attributes  established  for  subsystem  J  in 
the  input  data. 

Substep  5.  The  scores  for  all  attributes  for  the  Kth  subsystem  in 
the  comparison  systems  database  are  summed  and  the  sum 
is  multiplied  by  any  weighting  factor  for  the  input 
data  subsystem  that  has  been  input  by  the  user.  If  no 
differential  weights  are  supplied  by  the  user,  then 
all  subsystems  are  weighted  equally.  The  resulting 
value  is  the  figure  of  merit  for  the  Kth  comparison 
subsystem  in  relation  to  the  input  data  for  the  Jth 
subsystem. 

Substep  6.  The  process  described  in  substeps  3,  4,  and  5 

immediately  above  is  iterated  for  all  K  subsystems  in 
the  comparison  systems  database  that  are  comparable  to 
the  Jth  input  data  subsystem. 

Substep  7.  The  process  described  in  substeps  1  through  6  is 
iterated  for  all  J  subsystems  in  the  input  data. 

Help:  None.  This  is  an  autonomous  function  of  Product  Four. 


2.2  Production  of  principal  comparison  case. 

Rationale:  A  principal  comparison  case  is  produced  based  solely 
on  the  comparisons  and  figures  of  merit  resulting  from  the  initial 
activation  of  the  composite  comparison  engine. 

User:  This  action  is  user  independent. 

Product  Action:  Creates  a  principal  comparison  case  by  selecting 
'  the  comparison  subsystem  with  the  best  figure  of  merit  for  each 
subsystem  in  the  input  data. 

Help  Available:  None.  This  is  an  autonomous  function  of  Product 
Four. 

2.3  Identification  of  alternative  comparison  cases. 

Rationale:  The  identification  of  alternative  comparison  cases 
provides  helpful  information  to  the  user  during  the  next  function 
of  Phase  II,  in  which  the  user  can  accept  or  reject  the  principal 
comparison  case. 

User:  This  is  the  last  of  the  autonomous  functions  of  Phase  II. 

Product  Action:  Selects  alternative  comparison  cases  for  each 
subsystem  based  upon  the  figures  of  merit  associated  with 
alternative  comparisons.  The  figures  of  merit  for  each  subsystem 
are  selected  in  order  to  compose  alternative  comparison  cases. 

The  number  of  alternative  comparison  cases  described  by  the  most 
numerous  subsystem  in  the  comparison  systems  database  is 
developed . 

Help  Available:  None.  This  is  an  autonomous  Product  Four 
func  t ion . 

Step  2.4  User  review  of  principal  comparison  case. 

Rationale:  Before  proceeding  to  the  development  of  training 
characteristics  data,  the  principal  comparison  case  or  some  other 
comparison  case  must  be  accepted  by  the  user. 

Product  Action:  Presents  the  principal  comparison  case  for  user 
review.  Figure  8  shows  a  notional  screen  presenting  comparison 
case  data. 

User:  Reviews  the  principal  comparison  case  and  decides  whether 
or  not  to  accept  it. 

Product  Action:  When  the  user  selects  and  accepts  the  principal 
comparison  case,  product  function  proceeds  to  the  identification 
of  training  and  design  characteristics  (Phase  III).  When  the 
principal  comparison  case  is  rejected,  several  options  are 
presented  to  the  user  and  the  product  responds  accordingly.  The 
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COMPARISON  CASE  DATA 


For  New  System:  AHX  (Attack  Helicopter) 
Composite  identified  as  best  fit  to  inpot  data. 


Composite  Summary: 

Subsystem 

Comparison  Case 

Characteristics  Match  Data 

(new  system  comparability 
in  parentheses) 

Airframe 

AH- IT 

Weight  (  ♦  52) 

Crew  Size  (same) 

Range  (  +  152) 

Speed  (  +  102) 

Armament 

AH -64 

Missiles  (same) 

Gun  ( same ) 

Target  Acquisition 
and  Designation 

AH-64 

Direct  View  (same) 

LLLTV  (  +  102  resol.) 

IIR  (  +  252  resol.) 

Laser  Des .  (6  v  4  channel) 

Engines 

AH- IT 

Number  (same) 

SHP/Eng.  (  +  302) 

Communications 

AH-64 

UHF  (same) 

VHF  (same) 

HF  (same) 

Crypto  (same) 

Navigation 

AH-64 

VOR  (same) 

NAVSTAR  GPS  (same) 

Inertial  (new;  no  match) 

ASE 

AH-64 

IR  Detect  (  +  102  sens.) 
Laser  detect  (same) 

RF  Detect  (same) 

Fuel 

AH-64 

Type  (same) 

Capacity  (  +  82) 

PAGE  DOWN  for  more 

comparison  information 

Additional  poorer- 

fit  composites  :  2 

Options:  PAGE-DN 

PAGE -UP 

NEXT-COM  < CHANGE > 
CHK-PNT  SAVE -END 

PRINT-PG  PRINT -CM 

CHG- INPUT  DO- TNG  ACCEPT 

SAVE-DO  ABORT  HELP 

Change  the  composite  by  substituting  different  subsystem(s) 


Figure  8.  Comparison  case  data. 
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options  and  corresponding  product  functions  listed  below  are 
described  in  Steps  2.5  through  2.7: 

(1)  Review  the  alternative  comparison  case  candidates  (Step 
2.5). 

(2)  Override  Product  selections  with  respect  to  any  of  the 
generated  comparison  cases  (Step  2.6). 

(3)  Modify  input  data  and  rerun  comparison  case  identification 
(Step  2.7). 

Help  Available:  Explanations  of  the  options  available  and  the 
consequences  of  selecting  each  option  will  be  provided  as 
necessary . 

Default:  The  principal  comparison  case  is  accepted,  as  is  (the 
first  option). 

Step  2.5  Review  of  alternative  comparison  cases  and  modification  of 
the  principal  comparison  case. 

Rationale:  The  user  may  have  information  that  indicates  that  the 
principal  comparison  case  is  not  the  best  alternative  upon  which 
to  base  the  identification  of  the  training  and  design 
characteristics  data.  At  this  point  the  user  may  select  an 
alternative  composite  case  for  adoption  for  the  estimate. 

Product  Action:  Presents  list  of  alternative  comparison  cases 
generated  by  the  comparison  engine. 

User:  Selects  one  or  more  alternative  comparison  cases  for 

review,  and  examines  the  data.  The  user  can  accept  an  alternative 
comparison  case,  or  execute  one  of  the  other  available  options. 

Product  Action:  When  an  alternative  candidate  case  is  selected, 
the  Product  utilizes  the  alternative  for  generation  of  the 
training  characteristics  estimate. 

Help  Available:  Explanations  of  the  available  options  and  the 
implications  of  selecting  each  option  will  be  provided  as 
required. 

Step  2.6  User  override  of  Product  Four  comparison  case  selections. 

Rationale:  This  option  is  available  for  the  situation  where  the 
user  has  information  that  indicates  that  neither  the  principal 
comparison  case  nor  the  alternative  comparison  cases  are  suitable 
for  the  composite. 

User:  Overrides  principal  or  alternative  comparison  case 

subsystem  selections  and  modifies  the  comparison  case  at  will. 
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Modification  takes  place  by  the  user's  specifying  one  or  more 
alternative  subsystems  (from  the  comparison  systems  database)  to 
be  used  in  place  of  those  identified  by  the  comparison  engine. 

Product  Action:  Alters  the  comparison  case  to  reflect  the  user's 
mod  if icat ions . 

Help  Available:  Information  associated  with  applicable  subsystems 
in  the  existing  systems  database,  including  information  about 
their  evaluation  as  alternatives  for  developing  the  comparison 
case,  is  provided  as  required. 

Step  2.7  User  modification  of  input  data. 

Rationale:  At  this  point  the  user  can  select  to  revise  the  input 
data  on  which  the  generation  of  the  comparison  case  is  based. 

Product  Action:  Returns  the  user  to  Phase  I,  where  the  user  can 
revise  the  original  input  data.  The  Product  queries  the  user  as 
to  the  subsystem(s)  and  attribute(s)  to  be  re-entered. 

User:  Enters  the  information  in  a  manner  similar  to  original  data 
input . 

Product  Action  (after  user  action):  Saves  the  new  version  of  the 
input  data  and  returns  Product  function  to  activation  of  the 
comparison  case  identification  engine.  A  new  version  code  is 
added  to  the  new  version  data  for  archival  and  retrieval 
purposes . 

Help  Available:  All  of  the  help  that  is  available  during  Phase  I 
is  available  during  re-entry  or  modification  of  proposed  system 
input . 

Step  2.8  Data  management. 

Rationale:  Users  can  elect  to  save  or  to  delete  any  versions  of 
the  comparison  case  identification  outputs  or  input  data  sets,  to 
facilitate  additional  analyses  or  re-analyses  based  on  modified 
input  data. 

User:  Indicates  the  desire  to  delete  a  specific  version  of  the 
composite  comparison  or  data  input. 

Product  Action:  Automatically  saves  all  versions  of  the  input 
data  and  of  the  resulting  principal  comparisons  cases  and 
alternatives  and  assigns  version  numbers,  unless  directed  by  the 
user  to  delete  (a)  specific  version(s)  of  the  data. 

Help  Available:  At  any  point  previous  versions  of  data  can  be 
reviewed.  The  accomplishment  of  this  function  is  fairly 
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straightforward;  additional  user  help  is  not  expected  to  be 
re qu i red  . 


Default:  All  versions  of  comparison  case  identification  output 
and  data  input  are  saved. 


Phase  III  —  Training  and  Design  Characteristics 
Generation  and  Modification 


Overview 

The  flowchart  in  Figure  9  illustrates  the  functions  performed  in 
Phase  III.  After  the  user  accepts  a  comparison  case  (either  as 
generated  or  as  modified  by  the  user),  the  training  character¬ 
istics  and  design  identification  engine  is  invoked.  This  engine 
retrieves  the  training  and  design  characteristics  information 
associated  with  each  of  the  subsystems  in  the  comparison  case.  This 
information  is  then  conditioned  by  some  adjustment  factors  (to  be 
identified  in  the  detailed  design  of  Product  Four),  to  produce  the 
training  and  design  characteristics  for  the  proposed  system.  In 
addition,  remediation  and  prescription  data  associated  with  the  high 
drivers  and  design  problems  are  identified  during  this  Phase.  Once 
again,  the  user  has  the  opportunity  to  review,  modify,  and  override  the 
training  and  design  characteristics  and  other  data  before  accepting  the 
training  requirements  projections.  As  with  previous  phases  of  Product 
operation,  the  user  is  able  to  alter  input  data  or  the  comparison  case, 
in  order  to  explore  alternative  training  consequences  or  conduct 
sensitivity  analysis. 

Detailed  Functional  Description 

Step  3.1  Activation  of  training  and  design  characteristics  estimation 
engine . 

Rationale:  The  engine  is  activated  to  generate  the  training 
projection,  high  driver,  design,  and  remediation  prescription 
data . 

User:  In  order  to  activate  the  training  and  design  character¬ 

istics  estimation  engine,  the  user  accepts  a  comparison  case. 

Product  Action:  Retrieves  training  data  associated  with  each 
subsystem  included  in  the  composite. 

Help  Available:  This  is  an  autonomous  function  of  Product  Four, 
user  help  is  not  expected  to  be  required. 
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Step  3.2  Application  of  error  and  additivity  factors  to  training 
data . 

Rationale:  This  step  allows  for  consideration  of  the  range  of 
error  involved  in  this  type  of  estimate  and  of  times  when  the 
simple  addition  of  training  factors  will  give  an 
unreasonable  estimate  of  training  requirements.  This  function  is 
separate  and  independent  of  the  user  modification  of  the  data  that 
occurs  at  the  end  of  Phase  III. 

User:  It  is  expected  that  this  function  will  be  an  autonomous 

Product  action  and  will  occur  automatically  upon  activation  of  the 
training  and  design  characteristics  estimation  engine. 

NOTE:  The  overall  process  and  specific  data  manipulations 

underlying  the  application  of  error  and  additivity  factors  have 
not  yet  been  determined.  The  actual  process  will  depend  on 
findings  of  explorations  of  data  properties  in  later  Phases  of 
this  effort. 

Help  Available:  The  application  of  the  error  and  additivity 
factors  is  expected  to  be  an  autonomous  function,  as  such  user 
help  will  not  be  required. 

Step  3.3  Generation  of  training  projections. 

Rationale:  The  corrected  subsystem  training  characteristics  data 

are  used  to  generate  the  training  projections  by  subsystem  and  job 
that  are  part  of  the  output  of  the  Product. 

User:  This  is  an  autonomous  Product  Four  function. 

Product:  Develops  and  saves  the  profile  that  includes  training 

times  and  training  devices,  listed  by  operator  and  maintainer  job 
class  and  subsystem. 

Help  Available:  User  help  is  not  required. 

Step  3.4  User  review  of  the  training  projections. 

Rationale:  The  user  is  given  an  opportunity  to  modify  the 
training  projections  prior  to  output,  to  suit  cases  where 
additional  information  may  be  available.  This  accommodates  the 
application  of  constraints  upon  the  training  system  that  may  be 
imposed  as  a  part  of  the  system  requirements  generation  process. 

*Product:  Formats  data  for  user  review  and  presents  formatted 

data . 

User:  Reviews  the  training  projections  and  decides  whether  to 
accept  or  reject  the  training  projections.  A  notional  screen  of 
data  illustrating  user  review  is  shown  in  Figure  10. 
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TRAINING  SYSTEM  CHARACTERISTICS  ESTIMATE  DATA 
For  New  System:  AHX  (Attack  Helicopter) 


Operator  Training  Summary  Page  1  of  12 

Crew  Position  #1  (Pilot) 

Initial  Qualification  Training 
Comparison  Cases:  AH-1T,  AH-64 


Course  Description  Length  Hands-on  Academic  TDs  Other 


Initial  Entry 
Wing  -  Common 

Rotary 

Core 

14  wks 

8  wks 

6  wks 

2B24 

Ac  f  t 
TH55 
UH60 

Initial  Entry 
Wmg  -  Attack 

Rotary 

Track 

12  wks 

20  wks 

2  wks 

2B24 

2Fxxx 

Ac  ft 

AHX 

UH60 

Unit  Transtormation 
Qual if ication 

1 3  wks 

13  wks 

2Fxxx 

Ac  ft 
AHX 

Correction  Factors  Applied:  None 


PAGE  DOWN  for  more  training  information 

Options:  PAGE-L'P  MODIFY  ACCEPT  CHG-COMP  CHG-INPUT  CHK-PNT 

<PAGE-DN>  PRINT  HELP  SAVE-END  SAVE-DO  ABORT 

Display  Next  Page  of  Data 


Figure  10.  Training  system  c ha rac te r  i  s t  ic s  estimate  data. 


£1-43 


Product  Action:  When  the  user  accepts  the  training  projection, 
the  Product  proceeds  to  the  retrieval  and  presentation  of  the  high 
driver,  design  factors,  and  remediation  and  prescription  data. 

When  the  user  rejects  the  training  projection,  the  Product 
presents  the  following  options  that  are  described  in  Steps  3.5  and 
3.6: 

(1)  Modify  original  data  input  (Step  3.5). 

(2)  Modify  training  projection  (Step  3.6). 

Steps  3.5  and  3.6  are  not  performed  if  the  user  accepts  the 
training  system  characteristics  data  generated. 

Help  Available:  User  help  will  include  descriptions  of  the 
options,  reasons  for  selecting  each  option  and  the  consequences  of 
selecting  each  option. 

Default:  The  training  projections  are  accepted  without  user 

mod i f ic  a t i ons . 

Seep  3.5  User  modification  of  original  data  input. 

Rationale:  This  option  allows  the  user  to  perform  sensitivity 
analyses  to  explore  the  impact  on  training  system  characteristics 
of  different  input  data  or  defined  comparison  cases. 

User:  Indicates  subsystem(s)  of  comparison  case  to  be  modified. 

Product  Action:  Returns  user  to  original  data  input. 

Help  Available:  Any  help  available  during  the  original  data  input 
will  be  provided  here  as  required. 

Step  3.6  User  modification  of  training  projection. 

Rationale:  The  user  may  opt  to  directly  modify  the  projections 

provided  by  the  engine.  This  allows  for  cases  where  additional 
expertise  or  constraints  can  be  applied  to  refine  the  estimates. 

♦Product  Action:  Presents  training  profile  and  querries  user  as 
to  the  specific  data  to  be  modified. 

User:  Indicates  the  estimates  to  be  modifieu  and  alters  those 

estimates  as  desired.  The  user  also  has  the  option  of  documenting 
the  rationale  for  these  modifications. 

Product:  Creates  and  saves  another  version  of  the  projections  to 

reflect  the  user  modifications. 

Help  Available:  Prompts  and  querries  will  be  provided  as  required 
to  aid  the  user  in  accomplishing  the  modifications. 
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Step  3.7  Generation  of  high  driver,  design  characteristics ,  and 
remediation  and  prescription  information. 


Rationale:  This  information  is  part  of  the  final  output  of  the 
Product . 

User:  This  is  an  autonomous  Product  Four  function. 

Product  Action:  Retrieves  and  saves  high  driver  and  design 
characteristics  associated  with  each  subsystem  in  the  composite 
and  the  corresponding  remediation  and  prescription  data. 

Help  Available:  This  function  occurs  without  user  interaction; 
user  help  will  not  be  required. 

Step  3.8  User  review  of  the  high  driver,  design  characteristics,  and 
remediation  and  prescription  data. 

Rationale:  There  may  be  cases  when  the  user  desires  to  modify 
these  data. 

♦Product  Action:  Formats  and  presents  the  data  for  user  review 
and  modification. 

User:  Decides  to  accept  or  modify  the  data. 

Product  Action:  Upon  user  acceptance  of  the  data,  the  Product 
stores  either  the  unmodified  original  version  or  both  the  original 
version  and  the  user  modified  version.  If  user  decides  to  accept 
the  data  generated  originally,  Step  3.9  is  not  performed. 

Help  Available:  Help  will  include  explanations  of  instances  when 
user  modification  is  desirable. 

Default:  The  high  driver,  design  characteristics,  and 

remediation  and  prescription  data  are  saved  without  user 
mod i f icat ion . 

Step  3.9  User  modification  of  the  high  driver,  design  characteristics, 
and  remediation  and  prescription  data. 

Rationale:  The  user  may  elect  to  customize  these  data  or  to 
modify  the  data  based  upon  additional  information  available. 

♦Product  Action:  Presents  the  data  and  querries  user  as  to  the 
specific  data  to  be  modified. 

User:  Selects  the  data  to  be  modified  and  makes  changes  as 

desired.  Indicates  when  no  changes  are  to  be  made  or  when 
finished  with  changes. 
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Product:  Creates  and  saves  another  version  of  the  data  to  reflect 

the  user  modifications. 


Help  Available:  Querries  and  prompts  to  aid  in  the  modification 
process  will  be  provided  as  required. 

Step  3.10  Data  management. 

Rationale:  Both  the  original  output  from  the  training  and  design 
characteristics  identification  engine  and  version  resulting  from 
user  modifications  may  be  helpful  to  the  user  or  to  future  users. 

User:  Data  management  during  Phase  III  is  automatic;  the  user  is 
not  required  to  perform  activities  related  to  data  management 
other  than  indicating  data  or  files  to  be  deleted. 

Product  Action:  Automatically  saves  the  original  version  of  the 
training  and  design  characteristics  and  other  data,  and  any 
subsequent  versions  that  result  from  user  modifications  or 
re-analysis  of  input  data. 

Help  Available:  Data  management  is  an  autonomous  function  of 
Product  Four  with  the  exception  of  deleting  files;  information 
related  to  the  deletion  of  files  including  when  and  why  will  be 
provided  as  required. 


Output 


Ove  rv iew 


Once  the  user  has  made  the  desired  changes  to  the  data  (if  any), 
the  data  (and  results  of  any  mod  if icat ions )  are  saved  and  can  be  output 
at  will.  Output  data  sets  can  be  archived  at  will,  for  comparison  in 
sensitivity  analyses  or  as  templates  for  future  exercise  of  the 
Product.  Outputs  need  not  be  generated  at  the  end  of  each  use  of  the 
Product.  An  output  of  any  completed  version  of  analyses  can  be  called 
for  at  any  time.  An  overview  of  the  output  process  is  provided  by  the 
flowcharts  in  Figure  11. 


Detailed  Functional  Description 

Step  0.1  User  definition  of  desired  output. 

Rationale:  Various  output  formats,  as  well  as  different  versions 
of  data  to  be  output  will  be  available. 

Product  Action:  Querries  user  as  to  the  format  and  version  of 
data  to  be  output. 
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User:  Indicates  the  specific  information  to  be  output. 


Help  Available:  Help  will  include  lists  of  the  available  versions 
of  each  information  type  and  opportunity  for  review  of  that  data. 

Default:  The  latest  user  modified  version  of  the  training 
estimate  and  ancillary  data  are  selected  for  output. 

Step  0.2  Data  Formatting  and  Output. 

Rationale:  The  data  will  be  formatted  in  a  fashion  determined  to 
be  useful  by  the  Product  Four  designers. 

User:  Indicates  that  output  is  desired  (if  variable  formats  for 
output  are  identified  by  the  Product  Four  designers,  the  user  will 
select  an  applicable  format). 

Product:  Formats  and  outputs  data  as  indicated. 

Default:  A  default  format  option  will  be  defined  by  the  Product 

Four  designers. 

This  concludes  the  description  of  Product  Four  operation.  The 
next  section  of  this  paper  describes  the  data  required  for  developing 
and  operating  Product  Four,  and  where  those  data  will  be  obtained. 
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SECTION  FOUR 


PRODUCT  FOUR  DATA  REQUIREMENTS  AND  DATA  SOURCES 


Overv iew 


This  section  of  the  concept  paper  discusses  data  requirements  for 
Product  Four  operation  and  data  sources  from  which  information  will  be 
gathered  to  develop  Product  Four.  This  discussion  is  keyed  to  the 
first  four  of  the  five  sequential  steps  of  Product  function,  as 
discussed  in  the  previous  section.  The  final  step,  Output,  requires  no 
unique  data.  A  subsection  of  this  section  is  devoted  to  discussing 
data  requirements  for  each  of  the  steps  requiring  data.  In  each 
subsection,  data  requirements  are  discussed  first.  Then,  data  sources 
are  addressed  in  detail.  Note  that  significant  amounts  of  the  data 
required  to  develop  Product  Four  will  be  elicited  from  SMEs. 

Approaches  for  effectively  and  efficiently  eliciting  these  data  are 
addressed  in  appropriate  places  in  this  section.  Figure  12  presents  an 
overview  of  data  requirements  and  sources  for  the  data  needed  to 
develop  Product  Four. 


User  Activation  Data  Requirements  and  Sources 


Data  Requirements 

The  first  step  of  system  operation,  User  Activation,  requires  the 
user  to  decide  among  three  choices:  (1)  Develop  a  new  training 
estimate  for  a  new  system;  (2)  Review  and  possibly  refine  one  or  more 
existing  estimates  already  on  file;  or  (3)  Enter  the  embedded  training 
function  of  Product  Four,  The  only  database  required  to  support  this 
phase  is  an  audit  file  which  archives  all  previous  files  formulated. 
This  file  is  developed  and  maintained  by  the  Product  Four  software 
system. 

The  audit  file  will  allow  a  user  to  select  one  or  more  previously 
developed  files.  For  each  file  selected,  Product  Four  will  reproduce 
the  template  of  functional  requirements  originally  given,  identify  the 
comparison  case(s)  which  Product  Four  produced,  and  highlight  any 
modifications  which  the  prior  user  made  to  override  the  product's 
output.  Additionally,  Product  Four  will  provide  an  audit  of  training 
characteristics,  training  high  drivers,  high  dr iver-assoc iated  design 
factors,  and  remediation  and  prescription  data  provided  for  each  file 
developed . 
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FTXR  DATABASES  AfC  DATA 


Data  Sources 


The  audit  file  will  be  dynamic  in  that  it  will  grow  as  each  user 
decides  to  save  a  training  estimate.  Each  new  file  will  be  indexed 
according  to  the  general  class  and  subclass  (e.g.,  aviation  [attack]) 
of  system  which  it  defines.  One  issue  for  further  study  is  determining 
how  this  growing  database  will  be  managed.  It  is  possible  that  the 
number  of  training  estimates  on  file  could  be  so  large  that  a  user 
could  be  inundated  with  options.  Some  files  may  differ  substantially 
in  specific  subsystems  selected  for  the  comparison  case,  whereas  others 
may  differ  only  in  performance  specifications  of  subsystem 
characteristics  (input  data).  Product  Four  will  be  designed  to  provide 
a  new  user  with  a  simple  way  to  access  existing  audit  files 
selectively. 

If,  in  the  User  Activation  phase,  the  user  decides  to  obtain  an 
entirely  new  estimate  based  on  specifications  for  a  new  type  of  system, 
Product  Four  proceeds  to  Phase  One,  data  model  development. 


Phase  I  —  Development  of  Data  Model  and  Data  Input 


Data  Requirements 


Two  data  sets  are  required:  user  input  data,  and  the  comparison 
systems  database.  These  are  described  below. 

Input  Data.  The  Product  Four  user  inputs  performance  character¬ 
istics  for  the  system  for  which  the  training  estimate  is  desired. 
Performance  characteristics  are  defined  at  the  subsystem  level,  in 
terms  of  specific  attributes  of  performance.  The  attributes  will 
naturally  differ  from  one  type  of  subsystem  to  another,  and  will  vary 
across  systems  of  a  given  type. 

Input  data  must  contain  the  same  type  of  information  as  is 
available  for  systems  in  the  comparison  systems  database.  Although 
some  performance  characteristics  for  new  systems  will  be  different  than 
those  of  existing  systems,  the  types  and  units  of  measure  should  be  the 
same . 


Our  approach  for  Product  Four  requires  input  data  and  comparison 
systems  be  identical  in  units  of  measure.  However,  users  will  not  be 
expected  to  be  able  to  provide  as  exhaustive  a  description  as  is 
available  for  existing  systems.  Input  data  will,  therefore,  be  entered 
according  to  a  data  model  or  template  of  critical  performance  criterion 
areas  (comparability  attributes).  This  data  model  will  be  derived  from 
the  form  of  data  that  reside  in  the  comparison  systems  database.  This 
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approach  is  critical  to  the  ability  of  the  Product  to  automate  the 
identification  of  comparison  cases  to  support  training  predictions. 

The  specific  form  of  the  input  data  can  be  identical  to  the  output 
of  Product  One,  as  long  as  each  performance  criterion,  represented  as 
Product  One  output,  can  be  isolated  and  assigned  to  appropriate  data 
model  subsystems.  Product  Four  input  data  will  be  in  the  form  of  a 
functional  profile  of  the  desired  system's  characteristics . 

A  basic  assumption  of  the  ASA/SAIC  approach  is  that  any  new  system 
to  be  developed  will  fit  the  classification  of  Army  systems  by  the 
following  mission  areas  and  proponent  agencies. 


MISSION  AREA 

PROPONENT 

Close  Combat  (Heavy) 

U.S.  Army  Armor  Center,  Fort  Knox, 
Kentucky 

Close  Combat  (Light) 

U.S.  Army  Infantry  Center,  Fort  Benning, 
Georgia 

Aviation 

U.S.  Army  Aviation  Center,  Fort  Rucker, 
Alabama 

Air  Defense 

U.S.  Army  Air  Defense  Center,  Fort  Bliss 
Texas 

Combat  Support 

U.S.  Army  Engineer  Center,  Fort  Belvoir, 

Engineering  4  Mine  Warfare 

Virginia 

Combat  Service  Support 

U.S.  Army  Logistics  Center,  Fort  Lee, 
Virginia 

Fire  Support 

U.S.  Army  Field  Artillery  Center,  Fort 
Sill,  Oklahoma 

Nuclear,  Biological, 

U.S.  Army  Chemical  School,  Fort 

Chemical 

McClellan,  Alabama 

Command  4  Control 

U.S.  Army  Combined  Arms  Center,  Fort 
Leavenworth,  Kansas 

Comraun icat ions 

U.S.  Army  Signal  Center,  Fort  Gordon, 
Georgia 

Intelligence  4 

U.S.  Army  Intelligence  Center,  Fort 

Electronic  Warfare 

Huachuca,  Arizona 

This  classification  supports  the  organization  of  data  by  system 
class  (essentially,  the  Branch  identification)  and  subclass  (type  of 
system;  e.g.,  Aviation — Attack  helicopter,  etc.). 
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Another  basic  assumption  is  that  every  Army  system  can  be 
described  in  terms  of  an  aggregation  of  subsystems  with  corresponding 
performance  requirements  associated  with  subsystem  functions. 

Comparison  Systems  Data.  The  second  data  requirement  is  the 
comparison  systems  database.  The  overall  approach  in  Produ,  t  Four  is  a 
template-matching  task  which,  like  any  template  matching  task,  is 
affordable  only  if  it  operates  within  a  limited  domain.  If  the  domain 
is  limited,  as  it  is  in  the  case  of  Army  systems,  template-matching  can 
be  extremely  useful.  The  comparison  systems  database  provides  the 
domain  from  which  existing  systems  are  identified  and  corresponding 
s  :bsysteras  are  selected  to  form  the  composite  comparison  case  on  which 
the  training  estimate  is  based. 

The  comparison  systems  database  will  have  three  organizing  levels 
of  data  elements:  systems,  subsystems,  and  subsystem  attributes. 

Figure  13  illustrates  the  relationships  and  structure  among  these  data 
elements.  The  systems  level  of  the  database  will  incorporate  all  major 
systems  within  the  combat  and  combat  support  branches  of  the  Array. 

Based  on  our  projections,  this  should  consist  of  no  more  than  165 
unique  systems. 

Subsystem  attributes  are  information  upon  which  training 
comparisons  between  existing  and  new  systems  will  be  based.  This  means 
that  based  on  similarities  among  subsystem  attributes,  training  for 
existing  systems  is  predictive  for  the  new  system.  As  such,  defining 
the  exact  nature  of  these  attributes  is  critical  to  the  accurate 
matching  of  existing  subsystems  against  desired  subsystems. 

The  example  in  Figure  14  shows  that  for  the  target  acquisition 
subsystem  (of  Stinger),  the  optical  sight  has  a  1:1  magnification  and 
that  operator  procedures  are  characterized  as  complex.  ASA/SAIC  will 
similarly  catalogue  for  each  subsystem  of  every  system  in  the  Existing 
System  Database,  the  specific  values  (e.g.  horsepower,  range,  channel 
capacity,  sensitivity,  resolution,  etc.)  of  its  attributes. 

The  number  of  subsystem  attributes  contained  in  the  comparison 
systems  database  will  be  large,  but  not  insurmountable  within  the  scope 
of  the  effort.  Our  initial  estimate  of  the  total  number  of  subsystems 
attributes  in  the  database  breaks  out  as  follows: 

11  System  Classes  (Branches) 

x  6  System  Types  per  Class 

x  2.5  Systems  per  System  Type 

x  10  Subsystems  per  System 

x _ 5  Attributes  per  Subsystem 

8,250  Total  Attributes 
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STRUCTURE  FOR  COMPARISON  SYSTEMS  DATABASE 


(Shown  as  hierarchial,  may  be  associative  or  relational  as  developed) 


Number  of_Array_Branches  (1  time)  <n0> 

Branch  (n0  times) 

Number  of  Systera_types  (1  time)  -  <nl> 

System_type  (nl  times) 

System  type_narae  (1  time) 

Assoc iated_informat ion_for  system_type  (1  time) 

Number  of_Exeraplars_of_System_Type  (1  time)  -  <n2> 

Exemplar_data  (n2  times)  (System  type  exemplar) 
Exemplar_noraenc lature  (1  time) 

Number_of_Sub sys tem_leve l_at tributes  (1  time)  -  <n3> 

Subsystem  level__attribute  (n3  times) 

Subsyst’em_Level_Attributes_Nomenc  lature  (1  time) 
Number  of_SubsyItem_Level_Attribute_Characterist ics 

(1  time)  -  <n4> 


Characteristics  noraenc lature_data  (n4  times) 
Characteristic_nomenclature  (1  time) 
Characteristic_value_and_tolerances  (1  time) 
Explanatory_notes  (optional) 


Figure  13.  Candidate  database  structure  (warnier  notation)  for  the 
existing  systems  database  and  input  data. 
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Example  record  concents: 


Systera_type  :*  Air  Defense  Artillery  (Man  Portable) 
Exemplar_Noraenc lature  :■  Stinger  (MXXX) 

Subsystera_Level_Attribute  :m  Target  Acquisition 
Characteristic_Noraenc lature_l  :*  Optical  Sight 
Character ist ic_Value  :*  Unity  Power 
Characteristic_Noraenc  lature_2  Operator  Procedures 
Characteristic  Value  :*  Complex 


Explanatory_note  :*  <operator  procedures  complex 
because  set-up  of  launcher  and  sighting  require  many 
steps  to  be  remembered  and  performed  in  exact  sequence> 


Figure  14.  An  example  data  record  for  an  entry  in  the  existing 
systems  database. 
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It  one  were  to  assume  that  it  would  take  three  weeks  to  obtain 
subsystem  attributes  for  each  of  165  system  types  (or  165  systems), 
this  would  add  up  to  about  a  four  person-year  effort.  This  is  likely  a 
gross  overestimate  of  required  level  of  effort,  because  there  are  two 
Array  databases  which  provide  detailed  information  to  the  subsystem' 
level  for  all  fielded  systems  and  systems  in  the  Material  Acquisition 
Process  (MAP).  These  databases  and  methods  for  developing  the 
comparison  systems  database  are  discussed  below. 


Sources  of  Data  for  the  Comparison  Systems  Database 


The  comparison  systems  database  is  a  key  element  to  the  success  of 
Product  Four.  Its  size  will  determine  how  long  it  will  take  the  user 
to  estimate  the  probable  training  for  a  new  system.  Therefore,  the 
number  of  subsystems  and  attributes  per  subsystem  must  be  kept  at  a 
minimum. 

Three  sources  of  information  will  be  used  in  generating  the 
Existing  Systems  Database: 

1.  the  Sample  Data  Collection  (SDC)  database; 

2.  the  Array-Wide  Test  &  Evaluation  Database  (AWTEDB);  and 

3.  SMEs. 

The  SDC  database  and  the  AWTEDB  will  be  used  as  sources  of  system 
and  subsystem  performance  characteristics  for  existing  systems.  The 
SDC  database  is  developed  by  AMC' s  Materiel  Readiness  Support  Activity 
(MRSA)  and  contains  Reliability,  Ava i lab i 1 i ty ,  and  Maintainability 
(RAM)  and  some  performance  data  on  all  major  systems  currently  fielded. 
These  data  are  provided  at  the  subsystem  level.  For  systems  not 
currently  fielded,  but  in  the  Materiel  Acquisition  Process,  the  AWTEDB 
provides  both  RAM  and  performance  data  on  all  systems  tested. 

Figure  15  shows  a  schematic  of  how  we  will  develop  data  for  the 
comparison  systems  database. 

SDC  Data.  The  first  data  source  is  the  mandatory  SDC  program  (AR 
750  -  37)  established  by  Deputy  Chief  of  Staff  for  Logistics  (DCSLOG). 
Discretionary  SDC  programs  are  established  by  equipment  proponents  when 
properly  justified  and  approved  by  the  DCSLOG.  Mandatory  and 
discretionary  SDC  requirements  are  identified  after  the  awarding  of  a 
full  scale  development  contract,  and  these  are  incorporated  in  the 
initial  draft  Materiel  Fielding  Plan  (MFP).  Major  Command  (MACOM) 
concurrence  with  the  SDC  concept  defined  in  the  MFP  is  formally 
conveyed  through  the  signed  Materiel  Fielding  Agreement  (MFA). 

There  are  three  levels  of  SDC  data  collection.  These  are:  user 
participant;  proponent  directed  level  one;  and  proponent  directed  level 
two.  User  participant  data  collection,  in  accordance  with  the  approved 
Field  Procedures  Guide  (FPG),  requires  unit  personnel  to  record  system 
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Developing  the  comparison  systems  databas 


performance  data  on  standard  Army  forms  and  forward  these  data  to  a 
source  designated  by  the  proponent  activity.  Proponent  directed  level 
one  data  collection,  in  accordance  with  the  approved  FPG ,  requires  unit 
personnel  to  record  the  data  on  the  standard,  modified  standard,  or 
approved  unique  forms.  On-site  proponent  agency  members  or 
representatives  collect  these  data  forms.  They  also  obtain  additional 
information,  perform  quality  checks,  and  forward  data  to  sources 
designated  by  the  proponent  activity.  The  third  type  of  SDC  data 
collection  is  proponent  directed,  level  two.  In  accordance  with  the 
approved  FPG,  proponent  agency  members  or  representatives  record  data 
as  events  occur,  collect  SDC  data  forms  completed  by  unit  personnel, 
and  conduct  on-site  observations  and  verbal  inquiries.  Data  are 
assembled,  edited,  quality  checked,  and  forwarded  to  the  proponent 
activity.  The  level  of  detail  in  which  the  data  are  collected  is 
determined  by  the  importance  of  the  system  and  cost  constraints. 

AWTEDB  Data.  The  second  data  source  for  Existing  System 
information  is  the  AWTEDB.  The  AWTEDB  has  been  developed  to  meet  the 
needs  of : 

1.  Department  of  the  Army  staff; 

2.  Army  Materiel  Command  (AMC); 

3.  Training  and  Doctrine  Command  (TRADOC); 

4.  Operational  Test  Si  Evaluation  Agency  (OTEA); 

5.  their  subordinate  commands;  and 

6.  other  Army  agencies 

for  collecting  and  disseminating  in  a  timely  manner  test  incident 
information  on  high-visibility  systems  being  considered  for  possible 
acquisition  by  the  Army.  The  data  and  information  collected  originates 
at  : 

1.  Army  installations  where  testing  normally  takes  place; 

2.  Army  agencies  with  responsibility  for  materiel  acquisition; 
and 

3.  test  facilities  of  government  contractors. 

The  data  are  collected,  stored,  and  disseminated  at  a  central 
computer  located  at  Aberdeen  Proving  Ground  (APG),  Maryland. 

The  AWTEDB  .ontains  system  and  subsystem  performance  attributes. 
These  attributes  are  those  listed  in  the  Required  Operational 
Capability  (ROC)  for  each  system.  The  ROC  states  concisely  the  minimum 
essential  operational,  RAM,  and  technical  performance  factors  to  be 
achieved  by  a  new  system.  These  ROC  areas  are  evaluated  in  the 
technical  testing  of  a  hardware  system.  Data  on  testing  results  in 
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these  areas  is  included  in  the  AWTEDB.  This  database  contains  test 
incident  data  on  all  the  systems  and  subsystems  currently  in  the  MAP. 

SME  Data.  The  third  data  source  used  in  building  the  comparison 
systems  database  is  SME  opinion.  SMEs  provide  the  data  used  to  develop 
the  comparison  rules  that  support  the  template  matching  process  in 
developing  the  comparison  case.  SMEs  may  also  provide  performance 
parameters  on  systems  and  subsystems  needed  to  develop  a  complete 
comparison  systems  database. 

SMEs  will  be  identified  from  among  the  personnel  of  the  school 
directorates,  school  instructors  and  perhaps  students  attending  various 
courses  at  the  school.  With  some  special  coordination,  it  may  be 
possible  (if  necessarv)  to  have  personnel  in  Forces  Command  units 
provide  subject  matte"  expertise. 

Care  must  be  taken  when  obtaining  SME  opinions.  To  facilitate  the 
collection  of  SME  opinions;  ASA/SA1C  will: 

1.  Tell  the  SMEs  who  we  are,  what  our  mission  is,  and  how 
their  input  will  be  used. 

2.  Have  the  SMEs  be  interviewed  independently.  SMEs  will  not 
be  allowed  to  discuss  questions  and  answer  as  a  group. 
Strong  personalities,  in  a  group,  can  often  overwhelm 
issues  of  fact  and  influence  individual  responses. 

3.  Have  an  established  reference  for  structuring  SME  opinion. 
For  developing  comparison  rules  and  tolerances  for  the 
comparison  systems  database,  this  will  require  as  a  minimum 
that  we  present  the  SME  with  a  list  of  subsystems, 
attributes,  and  units  of  measure  for  the  attributes.  SMEs 
will  then  be  requested  to  identify  rules  that  can  be  used 
to  judge  when  a  particular  attribute  is  substantially 
similar  to  or  different  from  the  value  for  the  existing 
subsystem  and  attribute  in  question.  This  may  require 
substantial  probing  after  the  initial  SME  response  for  a 
comparison  attribute.  However,  we  are  prepared  to  probe  to 
the  extent  necessary  in  each  case  to  obtain  the  rules 
needed  to  build  Product  Four. 

4.  Assure  SMEs  that  their  input  will  be  used  only  for  study 
purposes  and  that  they  will  not  be  associated  individually 
with  their  responses. 

5.  If  the  number  of  attributes  for  which  rules  are  needed  for 
a  particular  class  of  systems  is  large  and  a  sufficient 
number  of  SMEs  is  available,  divide  the  data  gathering 
effort  into  sections  and  assign  SMEs  to  specific  sections. 
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The  number  of  SMEs  required  for  the  development  of  the  comparison 
systems  database  is  dependent  upon  how  much  or  how  little  can  be 
obtained  from  the  SOC  and  AWTEDB  databases.  The  SDC  database  is 
promising  because  it  is  arranged  in  a  hierarchical  level  of  subsystems 
in  a  descending  level  of  specificity.  This  provides  the  analyst  and 
interviewer  with  a  useful  tool  for  structuring  an  SME  interview.  For 
example,  the  SDC  listing  for  Che  AH-1  helicopter  can  be  presented  to  an 
appropriate  combat  developer  at  Fort  Rucker  as  a  stimulus  tor  obtaining 
subsystem  performance  criteria  for  Che  approximately  ten  subsystems 
listed . 

SMEs  to  support  development  of  the  comparison  systems  database 
should  be  system-knowledgeable  combat  developers  of  at  least  the  rank 
of  E-6.  SMEs  are  required  to  help  define  a  limited  set  of  subsystems 
(an  average  of  10  subsystems  per  system)  and  for  each  subsystem, 
identify  subsystem  attributes  (an  average  of  5  per  subsystems)  which 
are  particularly  sensitive  to  the  quality  and  degree  of  training 
provided.  The  following  is  a  first  order  estimate  of  SMEs  necessary  to 
support  development  of  the  comparison  systems  database: 


Aviation  20 
A i r  De f ense  6 
Armor  14 
Nuclear,  Biological,  Chemical  5 
Engineering  7 
F ie Id  Ar t  l  1  lery  6 
Infantry  11 
Intelligence  6 
Signal  4 
Quartermaster  3 
C2  6 


TOTAL 


88  SMES 


The  SMEs  will  be  used  not  only  for  developing  the  comparison 
systems  database  but  also  to  assist  in  developing  rules  for  driving  the 
comparison  process  between  new  systems  requirements  and  existing 
systems  capab  il  it ies .  The  rules  identified  by  SMEs  for  judging  whether 
the  attributes  of  subsystems  are  sufficiently  similar  to  be 
considered  the  same  are  the  rules  that  will  be  used  by  the  comparison 
identification  engine  to  develop  comparison  cases  for  training 
estimates  development. 


Phase  II  —  Developing  the  Comparison  Case 
Data  Requirements 

The  template-matching  engine  allows  automated  ident i f icat ion  of 
comparison  cases,  and  includes  rules  for  determining  if  input  data 
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match  the  characteristics  of  subsystems  in  the  comparison  systems 
database.  The  minimum  requirement  for  developing  these  rules  is 
information  on  the  most  appropriate  set  of  dimensions  on  which  the 
performance  of  given  subsystems  should  be  compared  to  input  data,  and 
the  criteria,  or  tolerances  which  should  apply  in  making  such 
judgments.  As  discussed  above,  these  rules  are  derived  from  SME 
judgments,  and  capture  those  judgments  in  a  knowledge  base  for  re-use. 
The  rules  are  contained  in  the  comparison  systems  database,  associated 
with  every  comparison  attribute  for  every  subsystem  of  every  system  in 
the  database. 


Data  Sources 

The  source  of  the  comparison  rules  will  be  the  personnel  who  serve 
as  SMEs  for  Product  Four  database  development.  These  SMEs  can  be  the 
same  88  SMEs  used  to  identify  subsystem  attributes  for  the  comparison 
systems  database.  The  SMEs  must  be  (in  the  aggregate)  knowledgeable 
about  all  attributes  of  the  subsystems  of  each  system  to  be  included  in 
the  database.  We  estimate  it  will  take  two  days  per  SME  (176  SME  days) 
to  gather  data  for  developing  the  comparison  systems  database. 

Our  intention  is  to  gather  these  data  through  structured 
interviews  with  SMEs  at  the  proponent  schools.  An  interview  protocol 
will  be  developed  and  tested  with  the  assistance  of  several  SMEs, 
focusing  on  some  sample  systems  and  subsystems.  Testing  (in  Task  2  of 
this  effort'  will  enable  us  to  discover  the  most  fruitful  ways  to 
elicit  SME  opinion  about  the  comparison  rules  to  be  developed.  Once 
the  interview  protocol  and  techniques  have  been  debugged,  the 
interviews  will  proceed  with  SMEs  at  each  of  the  proponent  schools.  We 
anticipate  that  the  development  of  these  data  and  the  construction  of 
the  comparison  case  identification  engine  will  be  the  most  challenging 
aspects  of  developing  Product  Four. 

Once  the  comparison  rules  (criteria,  ranges,  values,  or  tolerances 
for  comparison)  have  been  gathered,  they  will  be  used  to  implement  the 
comparison  systems  database. 


Phase  III--Devoloping  the  Training  Estimate,.  High  Driver, 
Design  Characteristics,  and  Remediation  and  Prescription  Data 


We  wi'l  develop  two  somewhat  distinct,  but  closely  related,  sets 
of  data  to  support  development  and  presentation  of  the  training 
characteristics  estimate  and  the  training  "high  drivers,"  design 
problems,  and  remediation  and  prescription  outputs  of  Product  Four. 
Since  these  data  will  come  from  diverse  sources,  the  training 
characteristics  database  requirements  are  discussed  separately  from  the 
"high  drivers,"  etc.,  data  in  this  subsection. 
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Data  Requirements.  Training  characteristics  data  are  needed  as 
part  of  the  Product  Four  resident  databases  to  support  estimates  of 
probable  training  characteristics  for  the  new  system.  Data  will  be 
provided  for  each  system  and  subsystem  represented  in  the  comparison  . 
systems  database.  This  will  allow  training  data  coverage  for  any 
combination  of  existing  subsystems  which  are  used  to  define  composite 
comparison  cases.  The  data  elements  required  to  compose  the  training 
characteristics  database  (for  each  system  in  the  database)  are  as 
follows: 

1.  Operator  training  time,  by  subsystem,  for  each  class  of 
operator  job  associated  with  the  system; 

2.  Maintainer  training  time,  by  subsystem,  for  each  class  of 
raaintainer  job  associated  with  the  system; 

3.  Breakdowns  of  the  two  elements  above,  in  terms  of  hands-on 
versus  academic  training  time;  and 

4.  Types  ard  numbers  of  training  devices  (TDs)  associated  with 
training  for  each  subsystem  of  the  system,  and  the  system 
at  large. 

Figure  16  shows  a  scheme  for  the  level  of  detail  of  data  to  be 
collected  for  each  system  in  the  comparison  systems  database.  This 
figure  shows  requirements  for  data  broken  out  in  three  categories: 
operators  versus  maintainers;  subsystems;  and  jobs.  Although  data  may 
not  fall  out  neatly  among  these  categories,  the  composite-building 
nature  of  the  Product  Four  approach  will  require  separate  data  for  each 
subsystem,  and  for  operator  versus  raaintainer  functions.  It  is  also 
recognized  that  training  time  for  the  total  system  will  almost  always 
be  less  than  the  sura  of  composite  subsystem  training  times.  This  means 
that  a  correction  factor  must  be  considered  when  estimating  total 
training  time  for  the  entire  system,  when  composite  comparison  cases 
are  used.  The  following  subsection  describes  the  sources  from  which 
the  data  elements  will  be  obtained.  Figure  17  is  a  schematic  showing 
how  we  will  develop  training  characteristics  data. 

Data  Sources.  The  training  characteristics  database  is  an 
extension  of  the  comparison  systems  database.  For  every  subsystem  in 
the  comparison  systems  database,  there  will  be  training  characteristics 
data  identified  for  all  operator  and  raaintainer  jobs  related  to  the 
subsystem.  As  each  subsystem  of  the  comparison  case  is  identified  in 
Phase  II,  a  new  building  block  is  correspondingly  added  to  the  training 
projection.  The  rest  of  this  section  addresses  how  training  data  will 
be  obtained  for  each  subsystem  in  the  comparison  systems  database. 
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Figure  16. 


Schematic  of  training  system  data  to  be  incorporated  in 
product  four  training  characteristics  database  (for  one 
system) . 
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Figure  17.  Developing  Che  training  characteristics . database. 
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After  an  intensive  search  of  existing  databases  and  information 
systems,  we  have  concluded  that  there  is  no  single,  comprehensive  data 
source  that  associates  training  characteristics  with  each  subsystem  of 
all  Army  systems.  Such  an  association  is  the  heart  of  Product  Four; 

i.e.,  relating  training  to  a  composite  of  subsystems  which  have  never 
before  been  combined  into  the  same  system.  Although  HARDMAN  experts 
who  must  also  deal  with  composite  systems  have  been  faced  with  the  same 
challenge,  no  attempt  has  yet  been  made  to  predict  composite  system 
training  based  on  an  additive  rule  which  considers  characteristics 
implied  by  each  subsystem.  Instead,  HARDMAN  experts  abandon  the 
composite  when  required  to  project  training,  and  make  projections  on 
the  basis  of  characteristics  of  the  single  closest  fitting  existing 
system. 

There  are  four  primary  sources  of  data  for  the  training 
characteristics  database.  These  are:  SMEs;  New  Equipment  Training 
Plans  (NETPs);  Plans  of  Instruction  (POIs)  for  existing  training;  and 
the  Army  Training  Requirements  and  Resource  System  (ATRRS).  These  data 
sources  overlap  to  a  certain  extent,  and  must  be  further  investigated 
during  the  next  Task  of  the  effort  to  determine  the  extent  of  the 
overlap  and  finalize  data  gathering  approaches.  However,  the  nature  of 
the  data  suggest  two  potential  methods  for  data  collection.  We  propose 
to  evaluate  the  costs  and  benefits  of  each  approach  in  a  trade  study  to 
be  conducted  in  Task  2  of  this  effort.  The  following  paragraphs 
describe  each  method. 

The  first  data  source  is  SMEs.  Using  SMEs  is  labor  intensive  and 
involves  a  significant  number  of  SMEs  from  the  schools  and  from  Project 
Manager  for  Training  Devices  (PM  TRADE)  (for  training  device  data). 

The  steps  in  the  use  of  SMEs  to  determine  training  times  for  a 
subsystem  are  summarized  as  follows. 

1.  Identify  all  courses  which  relate  to  training  operator  and 
maintenance  skills  for  each  existing  system  in  the 
comparison  systems  database.  (Data  sources:  training 
course  inventories  [from  schools];  New  Equipment  Training 
Plans ) 

2.  Tabulate  training  times  for  each  course.  (Data  sources: 
POIs,  ATRRS) 

3.  Interview  school  instructor  and  PM  TRADE  SMEs  and,  for  each 
subsystem  within  the  area  of  expertise  of  the  SME , 
determine  how  many  hours  of  training  time  would  be  required 
if  only  that  unique  subsystem  had  to  be  learned,  and 
identify  the  breakouts  between  hands-on  and  academic 
training  involved. 

Data  to  support  Step  1  above  will  be  obtained  from  DA  Forms  5316, 
(NETPs),  and  from  school  listings  of  courses.  The  Deputy  Chief  of 
Staff  of  the  Array  for  Operations  (DCSOPS),  the  Army  proponent  agency 
for  NETP's,  maintains  a  database  which  identifies  all  jobs  and  courses 
associated  with  any  system  fielded  within  the  past  seven  years,  and  any 
system  currently  in  the  acquisition  process. 
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Step  2  data  will  be  obtained  from  the  ATRRS  database,  which 
contains  an  extensive  database  on  training  characterist ics  associated 
with  all  courses  taught  by  or  for  Army  personnel. 

The  SME  interview  process  described  as  Step  3  above  will  require 
SMEs  to  think  about  training  from  a  perspective  which  is  foreign  to  the 
way  they  do  business.  This  is  particularly  true  of  maintenance  courses 
where  skills  must  be  taught  across  a  number  of  subsystems  of  various 
systems.  Unlike  RAM  databases  in  the  logistics  communities  where 
performance  data  are  collected  at  the  component  and  sub-component 
levels,  training  databases  contain  data  at  the  levels  of  functional 
areas,  courses  and  jobs. 

Because  the  above  method  is  foreign  to  SMEs,  the  protocol  for  SME 
interviews  must  be  sensitive  to  SME  uneasiness.  It  will  be  designed  to 
educate  the  SME  with  regard  to  the  purpose  of  the  interview,  and  should 
encourage  SMEs  to  help  make  it  a  better  process.  The  protocol  will  be 
developed  with  the  help  of  several  SMEs  and  will  undergo  iterations  as 
required . 

Current  estimates  of  the  number  of  SMEs  needed  to  assist  in  the 
development  of  the  training  characteristics  database  is  as  follows: 

PM  TRADE  -  5  SME  days 

11  Schools  -  3  SME  days  per  school. 

This  is  a  total  of  38  SME  days  from  all  sources.  Next,  we  discuss 
an  alternative,  less  SME-intens i ve  ,  method  of  obtain  training 
characteristics  that  we  will  explore  in  Task  2. 

An  alternate  data  source  for  obtaining  training  characteristics  at 
the  subsystem  level  is  the  partial  replacement  of  Step  3,  SME 
interviews,  with  the  database  supporting  the  NETPs.  This  database,  the 
Army  Modernization  Training  Automation  System  (AMTAS),  is  an  extremely 
powerful  tool  because  it  is  the  only  identified  source  of  information 
which  isolates  training  data  at  the  subsystem  level.  It  also 
represents  the  most  current  impact  of  high-technology  systems  on 
training  requirements  and  thus  increases  the  probability  of  providing 
"best  fit"  projections  to  meet  new  system  performance  criteria  input  by 
the  Product  Four  user.  It  is  also  less  prone  to  the  compounding  of 
error  which  occurs  every  time  the  artificially-derived  training 
estimates  (based  on  SME  interviews)  are  added  up  across  subsystems. 

The  potentially  negative  aspect  of  using  NETPs  as  a  data  source 
involves  comprehensiveness.  As  of  now,  it  is  uncertain  how  many  of  the 
comparison  systems  database  subsystems  are  represented  in  the  AMTAS 
database.  To  understand  this  issue  better,  a  brief  discussion  of  the 
AMTAS  is  required. 


The  NETP  (residing  in  the  AMTAS)  is  required  before  any  new  system 
or  product  improvement  (PIP)  is  fielded.  This  plan  must  address,  at  a 
detailed  level,  all  new  training  required  by  the  new  or  modified 
systems.  The  training  estimate  given  in  the  NETP  identifies  the 
specific  increment  or  delta  in  both  operator  and  maintamer  training 
which  the  modified  or  new  system  requires.  The  utility  of  the  NETP  in 
Product  Improvement  data  will  be  most  useful  to  Product  Four.  For 
example,  if  a  Product  Improvement  involves  only  the  upgrading  of  a 
tank  turret,  che  NETP  will  only  address  the  impact  of  the  new  tank 
turret  in  terms  of  increased  training  time,  etc.  Therefore,  the  AMTAS 
database  will  only  yield  subsystem-specific  training  impacts  if  there 
was  only  one  subsystem  involved  in  a  product  improvement.  With  over 
1,000  NETP's  represented  in  this  growing,  archival  database,  it  is 
likely  that  Product  Improvements  will  provide  significant  amounts  of 
training  related  data  at  the  subsystem  or  near-subsystem  level. 

NETPs  are  developed,  coordinated,  published,  and  distributed  by 
the  materiel  developer  for  each  item  of  significantly  modified 
equipment  for  which  training  is  required  (AR  350  -  35).  This  includes 
ancillary  items  such  as  training  devices.  A  NETP  should  include,  as 
applicable,  training  for  staff  planners,  developmental  and  operational 
test  personnel,  trainers,  supporters,  and  users.  The  NETP  covers  all 
training  aspects  of  the  equipment  from  procurement  or  development  and 
testing  through  production  and  fielding.  NETP  considers  the  following: 
similarity  to  oreviously  fielded  systems;  current  state  of  the  training 
base  to  support  the  new  equipment;  technical  complexity  of  the  new 
equipment;  fielding  rate;  effects  on  unit  readiness;  overall  training 
strategy  for  the  new  equipment;  planned  density  of  the  new  equipment, 
available  trainers  in  the  units  to  increase  training;  and  quality  of 
personnel  to  be  trained;  available  training  devices,  ranges, 
facilities,  and  training  materials;  environment  where  new  equipment  is 
to  be  fielded;  funding;  ammunition  to  support  NET;  and  sustainment 
training  following  fielding. 

Data  pertinent  to  the  development  of  Product  Four  contained  in  the 
NETP  include  the  system  and  subsystem,  the  Military  Occupational 
Specialty  (MOS)  associated  with  that  system  and  subsystem,  and  the 
training  time  affecting  each  MOS.  Once  the  MOS  decision  is  made,  the 
hours  needed  to  train  that  MOS  on  that  new  system  and  subsystem  is 
determined.  Those  affected  MOSs  are  listed  and  can  be  determined  in 
reviewing  the  plan  in  two  ways.  First,  the  training  base  established 
(tbe)  relates  to  a  particular  MOS  the  exact  number  of  training  hours 
associated  with  the  new  equipment  to  bring  that  particular  MOS  up  to 
speed.  Second,  if  hours  are  listed  under  the  column  "additional  skill 
identifier  (ASI),"  then  that  MOS  requires  the  indicated  hours  of 
training  to  bring  soldiers  up  to  speed  on  that  new  piece  of  equipment. 
If  there  is  a  blank  next  to  a  MOS,  then  no  additional  hours  are  needed 
to  train  them  on  the  new  piece  of  equipment.  Both  grade  (10,  20)  and 
level  of  maintenance  [organizational  maintenance  (OM),  direct  support 
( DS ) ,  and  general  support  ( GS ) ]  can  be  determined  by  reviewing  this 
plan. 
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Another  potential  source  of  data  for  the  training  characteristics 
database  is  the  ATRRS.  This  system  reports  the  availability  of 
training  resources.  These  resources  include  instructor  contact  hours 
(ICH),  cost,  and  support  equipment.  ODCSPER,  Force  Management 
Division,  is  the  proponent  of  ATRRS.  U.S.  Army  Information  Systems 
Command's  (USAISC-P)  computer  in  the  Pentagon  is  host  to  the  ATRRS 
system.  This  information  system  draws  information  from  and  provides 
information  to  four  levels  of  command:  Department  of  the  Army; 

Military  Personnel  Center  and  its  Reserve  Component  counterparts;  Army 
school  systems;  and  Army's  schools  and  training  centers  (AR  350  -  10). 

A  valuable  trade  study  that  we  intend  to  conduct  in  Task  2  is  to 
select  a  representative  existing  system  and  to  implement  both  of  the 
above  methods  for  identifying  training  characteristics  of  the  its 
subsystems.  This  process  could  be  repeated  on  other  systems,  and  a 
trend  might  be  ascertained  with  regard  to  the  reliability  and  vali  dity 
of  the  alternative  methrds.  Obviously,  if  both  proved  to  be  of  equal 
value,  the  less  SME-intens ive  method  (i.e.,  the  NETP  method)  will  be 
less  costly  and  more  valid,  and  therefore,  the  chosen  method. 


Training  "High  Drivers,"  Design  Characteristics,  and 
Remediation  and  Prescription  Data 

Data  Requirements.  Three  data  requirements  have  been  identified: 
training  "high  drivers,"  design  characteristics  related  to  high 
drivers,  and  remediation  and  prescription  data. 

Training  high  drivers  are  skills,  knowledge,  and  abilities  which 
require  the  most  training  to  ensure  soldier  competence.  They  are 
indicated  by  a  high  percentage  of  time  needed  for  training  and  by 
subjective  factors  of  difficulty  provided  by  experts  who  provide  the 
training.  These  "high  drivers"  will  be  identified  and  catalogued 
according  to  systems,  subsystems,  operator  or  maintainers ,  and  specific 
jobs . 


Design  characteristics  related  to  high  drivers  is  the  second  data 
requirement.  These  are  known,  correctable  flaws  in  the  design  of  the 
SSIs  of  existing  systems  which  cause  increases  in  training.  The  design 
characteristics  data  will  be  restricted  to  factors  which  are 
correctable  within  the  state-of-the-art  of  SSI  design. 

Remediation  and  prescription  data  is  the  third  requirement.  These 
data  are  provided  as  principles  (associated  with  specific  training 
"high  drivers"  and  associated  design  characteristics)  to  guide  system 
designers  in  their  choices  during  design,  to  minimize  training 
requirements . 

The  key  to  ensuring  the  utility  of  this  database  is  to  provide  the 
correct  level  of  guidance.  Too  many  guidelines  may  unnecessarily 
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constrain  the  designer  or  be  so  specific  as  to  increase  the  probability 
of  inaccuracy,  and  thus  challenge  the  credibility  of  the  system  (this 
Product)  used  to  develop  the  guidance.  Too  few  guidelines  will  result 
in  a  high  repetition  of  over-general  statements  across  subsystems, 
which  will  quickly  be  ignored  by  the  user  and  designer.  The 
guidelines,  therefore,  must  be  responsive  to  the  actual  problems  that 
exist  with  systems  and  subsystems  in  the  comparison  systems  database. 

We  will  develop  these  data  within  two  specific  constraints: 

1.  The  work  to  develop  the  training  characteristics 
estimation  capability  of  Product  Four  will  not  be 
compromised  in  any  way  by  providing  these  data;  and 

2.  All  output  of  this  feature  will  relate  directly  to  training 
aspects  of  SSI  design  which  have  been  shown  to  be  problem 
areas  for  existing  systems.  This  means  that  Product  Four 
will  not  be  a  general  interface  design  guideline  aid. 

Note  that  we  do  not  expect  that  there  will  necessarily  be  training 
"high  driver"  or  design  problem  issues  with  each  subsystem  of  every 
system  to  be  explored  for  Product  Four.  In  fact,  it  is  probable  that 
only  a  relatively  small  percentage  of  systems  will  have  such  problems. 
However,  we  feel  it  is  very  important  to  provide  designers  with  a 
means  of  reducing  the  training  impact  of  design  decisions.  Thus,  we 
intend  to  pursue  the  development  of  this  (not  strictly  required) 
component  of  Product  Four.  Figure  18  summarizes  our  approach  to 
developing  these  data  for  Product  Four. 

Data  Sources.  Data  sources  for  critical  incidents  which  suggest 
design  problems  and  training  "high  drivers"  are:  the  SDC  database, 
the  AWTEDB ,  and  the  Work  Order  Logistic  File  (MDLF).  The  databases  are 
intended  to  be  used  as  sources  of  critical  incidents,  which  will  then 
be  validated  and  expanded  by  StfEs.  The  databases  are  described  below. 
Data  sources  for  remedial  design  guidelines  are  standard  human  factors 
re  ferences . 

The  SDC  program  (AR  750  -37)  was  described  earlier.  The  SDC 
database  is  arranged  in  a  hierarchical  level  of  subsystems  in  a 
descending  level  of  specificity.  Data  can  be  retrieved  at  any  level 
within  this  hierarchy.  This  database  is  used  in  this  Phase  of  Product 
Four  because  it  includes  failure  incidents.  Data  on  the  number  of 
incidents  of  equipment  failure  is  accessed  at  the  subsystem  level. 

This  number  is  an  aggregate  of  ail  incidents  which  have  occurred  at  or 
below  the  accessed  level.  The  assumption  is  that  hardware  failures  are 
frequently  due  to  inadequate  training  or  poor  human  factors 
engineering.  Hardware  which  can  be  easily  operated  and  maintained  by 
the  soldier  does  not  induce  errors. 

Operator  and  maintainer  induced  incidents  relating  to  subsystem 
failure  can  be  accessed  by  code;  e.g.,  #979  if  incident  was  caused  by 
maintainer  and  #437  if  the  incident  was  cause  by  an  operator.  These 
induced  errors  can  be  used  to  solicit  SME  opinion  to 
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Figure  18.  Developing  Che  training  "high  drivers,"  design  problems, 
and  remediation  or  prescription  database. 
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determine  if  Che  error  could  be  attributed  to  training  or  design 
factors . 

The  AWTEDB  was  described  earlier,  and  will  be  useful  for 
developing  training  "high  driver"  and  design  factors  data  to  support 
this  element  of  Product  Four. 

The  largest  source  of  data  in  the  AWTEDB  comes  from  technical 
testing  (TT)  Test  Incident  Reports  (TIR).  Some  operational  testing 
(OT)  TIRs  coming  from  OTEA  and  others  submitted  by  TRADOC  and  its  test 
agencies  are  also  part  of  this  database.  All  TIRs  are  standardized 
across  users.  The  basic  requirement  is  that  all  originators  of  data 
and  all  automated  systems  be  capable  of  transmitting  TIRs 
electronically  over  the  MILNET. 

The  second-most  high-volume  report  in  the  AWTEDB  is  the  Corrective 
Action  Report  (CAR).  CARs  are  originated  by  Army  project  managers  and 
developing  agencies,  and  are  sent  to  the  central  computer,  either  in 
hard  copy  or  e lec c ron ica 1  ly  ,  with  the  latter  preferred.  Once  in  the 
data  base,  CARs  are  interfaced  with  the  TIR  summaries  to  which  they 
pertain,  and  the  TIRs  which  have  been  deemed  to  have  been  corrected  by 
the  CARs  are  closed  out. 

The  AWTEDB  data  useful  to  Product  Four  will  be  limited  to  a  subset 
of  the  data  contained  on  the  TIRs.  This  subset  will  be  those  subsystem 
test  incidents  in  which  operator  or  maintainer  performance  errors 
occurred.  These  types  of  errors  are  often  attributed  to  a  lack  of 
sufficient  training  or  poor  human  factors  engineering.  These  are 
called  induced  errors.  The  CARs  pertinent  to  these  types  of  errors 
will  also  be  examined  to  determine  the  type  of  corrective  action  taken, 
if  any. 

The  WOLF  database  provides  maintenance  data  at  the  general  and 
direct  support  levels  (GS/DS).  The  maintenance  data  include  the 
following:  mean  downtime,  mean  time  to  repair,  and  man-hour  costs  by 

MOS.  The  WOLF  provides  these  maintenance  data  for  the  entire  life 
cycle  of  selected  items  of  each  unit.  Incidents  of  equipment  failure 
due  to  operator  or  maintainer  error  can  be  tracked  over  a  longer  period 
of  time  than  SDC  data.  This  data  source  is  useful  for  Product  Four  if 
system  comparability  needs  to  access  systems  not  currently  held  on  the 
limited  basis  as  in  the  Array-Wide  Test  and  Evaluation  Database  or  the 
SDC  Database. 

The  items  chosen  for  inclusion  in  this  database  are  listed  in  AR 
220  -  1  (Unit  Status  Reporting).  Items  contained  within  this 
regulation  are  considered  critical  to  Array  unit  function.  Without  these 
items  the  unit  would  be  unable  to  perform  its  mission.  The  WOLF 
provides  a  broad  picture  of  what  is  happening  to  these  end  items  in 
DS/GS  maintenance  units  based  on  a  monthly  volume  of  about  20,000  work 
orders.  This  system  became  operational  in  May,  1985.  The  most  current 
12  month  data  is  kept.  Each  month  the  oldest  month  drops  out  ss  the 
newest  month  is  added  on.  The  WOLF  provides  incident  data  from  which 
to  glean  training  high  drivers. 
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Product  Four  will  make  use  of  the  comprehensive  and  detailed 
lessons  learned  available  in  the  several  Army  databases  discussed  above 
to  develop  data  to  make  up  the  training  "high  drivers"  and  design 
factors  database.  The  key  to  this  approach  is  Che  use  of  Army 
documented  incidents  as  discussion  items  with  training  developer  SMEs. 
For  those  subsystems  of  the  composite  system  which  have  been  desigrated 
as  critical  in  prior  tests,  we  will  present  all  reports  identifying 
human-induced  failures  to  SMEs  for  evaluation. 

The  first  step  in  SME  interviews  will  be  to  show  each  SME  the  list 
of  incidents  for  those  subsystems  pertinent  to  the  SME's  area  of 
expertise.  The  SME  will  be  asked  to  comment  on  the  validity  of  these 
reports  based  on  the  SME's  own  experiences.  Each  SME  will  then  be 
asked  if  additions  to  Che  list  of  critical  incidents  should  be  made 
for  each  subsystem. 

When  a  complete  list  of  problem  areas  is  finalized,  the  SMEs  will 
be  asked  if  the  quality  or  amount  of  training  contributed  to  the 
problem,  or  whether  engineering  design  was  primarily  at  fault.  The 
interviewer  will  then  attempt  to  identify  ways  to  either  redesign  the 
system  or  improve  the  training  approach.  All  information  elicited  in 
SME  interviews  will  be  recorded  in  detail  to  support  database 
deve lopraent . 

After  the  training  "high  drivers"  and  design  problems  information 
is  elicited  from  the  SMEs  during  interviews,  the  data  will  be  recorded 
in  the  database  structures  of  Froduct  Four,  associated  with  specific 
subsystems  of  systems  in  the  comparison  systems  database.  Where  we  are 
able  to  identify  effective  and  straightforward  SSI-  design 
remediation  or  prescription  data,  these  will  be  added  to  the  records 
associated  with  the  other  data  for  each  subsystem,  to  facilitate 
generation  of  outputs. 
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SECTION  FIVE 


TIME  REQUIRED  TO  USE  PRODUCT  FOUR 


We  currently  estimate  that  the  Product  can  be  exercised  through 
one  iteration  in  about  four  to  six  hours  of  clock  time.  This  assumes 
that  the  user  does  not  radically  alter  the  input  data  model.  It  also 
assumes  that  the  user  does  not  conduct  extensive  explorations  of  the 
intermediate  Product  outputs  (comparison  case  and  training  profile),  or 
perform  sensitivity  analyses.  Radically  modifying  the  input  data 
model  may  add  several  hours,  perhaps  a  day,  to  the  time  required  to  use 
Product  Four.  Tnis  might  be  required  if  a  new  type  of  subsystem  must 
be  added  to  the  data  model,  or  if  the  attributes  for  many  of  the 
subsystems  for  developing  the  estimate  must  be  changed. 

The  user  can  extend  the  time  required  to  use  Product  Four  by 
closely  scrutinizing  the  intermediate  products  produced  in  operating 
Product  Four  and  modifying  them  in  various  ways.  We  are  uncertain  how 
much  time  this  might  involve,  but  it  could  amount  to  between  hours  and 
days,  depending  on  the  inclinations  of  the  user. 

The  user  can  also  extend  the  time  taken  to  develop  a  training 
characterist  ics  estimate  by  performing  sensitivity  analyses.  We  expect 
that  each  iteration  of  Product  Four  operations  in  a  sensitivity 
analysis  could  require  from  one  to  as  much  as  three  hours,  depending  or 
how  much  of  the  input  data  or  the  comparison  case  the  user  chooses  to 
modify.  For  a  sensitivity  analysis  involving  four  or  five 
alternatives,  perhaps  one  day  would  be  required.  The  amount  of  time 
will  go  up  more  or  less  linearly  with  the  number  of  alternatives  the 
user  chooses  to  explore. 

The  time  required  to  gather  and  compose  input  data  for  using 
Product  Four  is  unknown  at  this  point.  If  Product  One  has  been 
exercised  (and  if  it  produces  the  level  of  detail  of  system  performance 
characteristics  that  wj  expect),  the  user  will  simply  have  to  gain 
access  to  that  data.  We  do  not  think  this  will  be  the  typical  case, 
however.  It  is  more  likely  that  much  less  detailed  data  (e.g.,  from  a 
Mission  Area  Development  Plan  [MADP]  or  similar  documents)  will  be 
available.  In  this  case,  either  the  user  will  have  to  rely  on  defaults 
from  the  Product  Four  existing  systems  database,  or  will  have  to  access 
SMEs  to  augment  the  available  data.  If  defaults  are  relied  on,  little 
time  will  be  required.  But,  if  SMEs  must  be  consulted  for  performance 
characteristics  estimates,  much  more  time  couid  be  required.  There  is 
presently  no  way  to  estimate  the  amount  of  time  users  may  need  to  get 
SME  inputs  for  the  input  data. 
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SECTION  SIX 


DEVELOPING  PRODUCT  FOUR 


Product  Four  will  be  developed  in  two  stages,  corresponding  to  the 
remaining  two  Tasks  of  the  overall  effort.  The  following  discussion 
details  the  activities  that  we  will  perform  in  each  of  the  two  tasks  to 
ensure  a  completely  functional  Product  Four  at  the  end  of  the  contract 
period . 


Stage  1  —  Design  Specification  Development 


In  Task  2,  we  will  concentrate  on  developing  detailed  design 
spec  i  f  icat  ions  for  the  Product.  The  preparation  of  the  design 
specification  will  involve  consideration  of  exactly  how  to  implement 
che  Product,  as  well  as  development  of  specific  data  types,  algorithms, 
and  processes  to  be  included.  This  constitutes  the  detail  design  of 
software  and  databases  to  implement  the  Product  concept .  We  intend  to 
adopt  a  software  design  approach,  as  well  as  allied  documentation 
requirements,  for  development  of  the  design  specification.  We 
anticipate  the  ultimate  specification  will  be  formatted  as  a  type  B5 
(per  MIL-STD-490)  computer  program  development  specification, 
accompanied  by  additional  highly  complete  information  describing 
training  and  interface  characteristics. 

The  specification  development  process  will  begin  by  identifying 
all  data  elements  required  for  effective  operation  of  the  Product,  in 
exhaustive  detail.  Databases  will  then  be  designed  with  record  and 
element  structures  implemented  to  efficiently  support  the  execution  of 
all  software  and  user  interface  processes  involved  in  Product 
operation.  Data  specifications  will  be  developed  (and  documented  per 
Warnier  methods)  prior  to  any  processing  design,  so  that  an  efficient 
implementation  and  test  approach  can  be  adopted.  A  sizing  effort  for 
total  database  structures  (above  and  beyond  what  has  been  done  to  date) 
will  also  be  conducted,  in  order  to  enable  estimates  of  hardware 
support  requirements  for  Product  Four. 

Next,  the  User-System  Interface  (USI)  for  the  Product  will  be 
specified,  along  with  development  of  the  embedded  training  approach  for 
product  familiarization  and  support.  We  currently  anticipate  using 
menu-driven  im  'face  protocols  to  the  extent  that  is  possible,  in 
order  to  develop  a  flexible  and  easily  modifiable,  but  efficient  mode 
of  interaction  for  product  users.  A  menu  interface  structure  is 
extremely  efficient  from  a  learning  to-use  standpoint,  and  also  is 
relatively  simple  to  implement.  Embedded  "helps"  for  all  functions 
and 
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interactive  modes  of  the  system  will  be  designed  into  the  USI.  An 
embedded  training  mode,  separate  from  normal  system  operations,  will 
also  be  included  to  facilitate  the  ability  of  first-time  users  of  the 
Product  to  quickly  "come  up  to  speed"  and  be  able  to  develop  training 
characteristics  estimates  quickly. 

Design  of  outputs  will  be  the  next  step  in  the  development  of  the 
functional  specification.  We  presently  have  an  excellent  idea  of  the 
information  which  will  be  developed  by  the  Product  at  both  intermediate 
stages  of  the  estimation  process  and  as  final  results,  as  discussed  in 
Section  Two  of  this  paper.  We  will  structure  the  information  for 
output  at  each  stage  such  that  it  presents  the  level  of  detail  needed 
by  each  class  of  user,  including  the  specific  information  on  which  the 
user  must  act,  in  a  clear,  concise,  usable  form.  All  data  elements 
will  be  explicitly  labeled,  and  the  principles  set  forth  by  Smith  and 
Hosier  (1984)  will  be  adhered  to  in  the  design  of  both  the  USI  and 
output  formats. 

The  next  activity  will  be  a  top-down  functional  decomposition  of 
the  required  processes  of  Product  operation,  to  develop  an  overall 
modular  design  for  software  and  procedures  which  will  be  part  of  the 
Product.  We  anticipate  limiting  the  decomposition  at  this  point  to 
complete  definition  of  the  major  processes  of  Product  operation,  with 
the  operational  details  left  for  Task  Three  development. 

Once  the  high-level  operational  modules  of  the  system  are 
described  through  the  hierarchial  decomposition  process,  an 
Input-Process-Output  (IPO)  analysis  will  be  conducted  for  each  modular 
element  of  the  system.  This  analysis  will  identify,  in  detail,  the 
input  requirements  of  each  module,  the  processing  which  will  take  place 
in  the  implemented  module,  and  the  outputs  of  the  module.  Inputs  and 
outputs  which  are  operator-provided  will  be  explicitly  referenced  to 
USI  and  output  formats,  as  necessary.  Internal  process  Input/Output 
(1/0)  elements  will  be  referenced  to  specific  database  elements. 
Processes  will  be  described  in  the  maximum  detail  possible,  and 
documented  in  Chapin  chart  format  for  consistency  and  to  implement  an 
audit  trad  for  later  actual  development  of  the  product. 

After  detail  development  of  the  design  specification  is  complete, 
we  will  prepare  an  overall  summary  of  the  design  to  support  ARl's 
review.  This  summary  will  include  data  security  cons iderat ions 
(especially  if  remote  access  is  to  be  utilized),  and  will  discuss  in 
detail  how  institutional  acceptance  of  the  product  w’ ' *  he  gained. 

This  latter  point  will  be  an  expansion  of  the  apT'  .o  institution¬ 
alization  of  the  Product  presented  ir>.  Section  Nine  this  paper. 

The  final  activities  in  this  phase  will  be  exhaustively  document 
the  results  of  the  design  spec i f icat ion  development  effort.  The  design 
specifications  will  be  documented  such  that  Che  following  critical 
elements  are  presented  clearly  and  concisely  in  the  h-«ic  specification 
format : 


El-75 


1.  Required  data  inputs  for  the  Product  as  a  whole  and  for 
each  functional  element,  the  location  of  the  data,  and  how 
data  will  be  accessed  in  developing  the  Product  in  the  next 
stage  of  the  effort. 

2.  The  components  of  the  Product  (in  terras  of  hardware  ana 
software  to  be  included),  how  each  will  be  obtained,  and 
how  they  will  interact  in  the  ultimate  Product. 

3.  Processes  by  which  the  Product  will  produce  its  outputs 
(the  detailed  B5  specification  material). 

4.  Detailed  description  of  Product  outputs,  as  defined  in  the 
analyses  discussed  above.  Both  interraed iate  products  and 
final  products  will  be  described  exhaustively  in  terms  of 
content,  format,  and  access. 

5.  Computer  file  security  assurance  measures  to  be 
incorporated  in  the  detailed  development  of  the  Product. 

6.  Means  to  be  taken  to  insure  organizational  acceptance  of 
outputs  of  the  Product. 

7.  A  detailed  schedule  and  estimate  of  costs  for  Product 
deve lopment . 


Stage  2  —  Product  Development 


In  the  second  stage  of  the  effort,  conducted  in  Task  3,  Product 
Four  will  be  developed,  debugged,  and  tested  with  realistic  estimation 
problems.  Also,  iterative  reviews  with  users  will  take  place,  to 
identify  issues  or  problems  to  be  resolved  (in  product  usability,  USI, 
or  product  content  and  format).  The  following  activities  will  take 
place  during  the  development  phase. 

First,  we  will  identify  the  hardware  needed  for  Product  Four 
implementation,  and  access  the  needed  hardware  resources.  We  do  not 
expect  that  large  scale  computational  resources  will  be  required  to 
implement  this  product.  We  anticipate  that  a  machine  on  the  close 
order  of  an  IBM  PC/AT  (or  compatible  machine)  will  be  adequate  to 
support  the  software  and  data  requirements  for  this  Product.  Since 
this  type  of  personal  computer  resource  has  become  the  second- 
generation  PC  standard  for  DoD  at  the  present  time,  there  is  a  high 
likelihood  that  such  machines  will  be  available  to  the  user 
communities.  We  do  not  anticipate  that  hardware  compatibility  will  be 
a  problem. 

I’e  propose  to  purchase  such  a  machine  for  use  in  developing  the 
Product;  this  machine  will  of  course  later  revert  to  the  Army  for  use 
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by  the  Product's  users.  At  the  same  time,  suitable  software  will  be 
purchased  and  installed  on  the  target  machine,  including  operating 
system,  utility  programs,  and  language  compilers. 

Next,  we  will  undertake  detailed  software  design.  The  top-level 
design  produced  in  specification  development  will  be  continued  until 
software  is  defined  down  to  the  unit  function  level.  A  development 
plan  for  the  software  will  then  be  prepared  and  implemented  to  support 
comprehensive  management  of  the  software  development  process. 

Next,  we  will  develop  and  debug  the  software.  We  expect  that  a 
minimum  of  two  iterations  of  development,  testing,  and  debugging  will 
take  place.  The  first  iteration  will  occur  within  the  project  team, 
and  will  concentrate  solely  on  the  development  of  valid,  well- 
documented  software  which  functionally  executes  the  processes  we 
specify  in  the  system  design.  This  stage  will  culminate  with 
comprehensive  user  tests  of  the  software  (using  test  databases),  to 
identify  problems  with  the  USI  or  with  outputs.  The  second  iteration 
will  concentrate  on  optimizing  the  USI  and  output  characteristics ,  and 
on  testing  the  functionality  of  the  Product  against  successively  more 
comprehensive  databases.  In  this  process,  we  will  attempt  to  exercise 
all  possible  cone ingenc ies .  We  will  evaluate  the  consistent 
functionality  of  the  Product  and  the  usability  of  outputs  in  the  wide 
range  of  situations  likely  in  real-world  application  of  the  Product. 

Concurrent  with  the  development  and  testing  of  the  software  for 
Product  Four,  we  will  develop  the  data  required  for  the  Product's 
databases.  We  will  conduct  the  data  development  approaches  discussed 
in  Section  Four  of  this  paper  to  develop  each  of  the  major  databases  in 
detail.  As  the  data  are  gathered,  they  will  be  formatted  according  to 
our  database  design,  and  incrementally  added  to  the  master  database 
files.  All  second-phase  user  and  function  tests  of  the  Product  will  be 
conducted  against  the  master  database. 

Next,  we  will  develop  and  implement  the  embedded  training 
component  of  Product  Four.  Although  we  here  call  this  out  as  a 
separate  activity,  development  of  embedded  training  is  most  likely  to 
occur  in  parallel  with  software  development  and  testing.  The  embedded 
training  component  will  be  iteratively  refined  along  with  the  software, 
to  ensure  that  the  training  required  by  users  is  in  fact  present  in  the 
ultimate  system. 
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SECTION  SEVEN 


PRODUCT  FOUR  USER  TRAINING 


The  characteristics  of  the  users  we  have  identified  for  Product 
Four  suggest  that  users  will  not  be  knowledgeable  in  the  techniques 
that  underlie  Product  Four  functioning.  Further,  it  is  expected  that 
users  will  know  very  little  about  database  management,  software,  or 
computer  use.  Since  we  will  be  unable  to  establish  a  training  element 
for  Product  Four  users  (by  restriction  in  the  contract  SOW),  the 
Product  will  have  to  provide  all  training  for  its  users. 

We  propose  to  build  into  the  software  and  databases  of  Product 
Four  two  components  that  will  enable  users  to  learn  and  use  the 
capabilities  of  Product  Four.  The  first  is  a  comprehensive  ET 
capability.  The  second  is  a  context-sensitive  Help  and  Explanation 
capability.  We  anticipate  that  the  two  components  will  share  many  data 
elements  and  software  routines,  since  their  purposes  and  functions  are 
basically  similar. 


Embedded  Training  Capability 


The  ET  capability  will  be  accessed  from  the  operating  system 
level.  A  unique  command  will  be  provided  to  call  up  and  begin  the  ET 
component,  separate  from  normal  Product  functions.  This  capability 
allows  "off-line"  training  to  prepare  new  users  of  the  Product  to  learn 
its  functions  and  capabilities,  as  well  as  review  or  sustainment  for 
more  experienced  users.  The  Product  Four  ET  component  will  contain  the 
following  functional  capabilities: 

1.  Modular  Lessonware:  Specific  topics  will  be  organized  into 
lessons,  which  can  be  taken  independently  by  a  user.  An 
overall  syllabus  structure  will  guide  initial  training,  but 
the  user  will  not  be  constrained  to  take  the  training 
modules  in  any  specific  order.  Following  of  the  syllabus 
structure  for  new  users  will  be  encouraged,  however.  We 
presently  anticipate  the  following  major  lesson  topics  for 
the  syllabus: 

a.  Basics  I  -  Introduction  to  the  Training 

Characteristics  Estimation  Method  and  Software 
System 
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b.  Basics  II  -Input  Data  Requirements  and  Data  Input 
Prac  t ice 

c.  Basics  III  -  Understanding  and  Interpreting 
Comparison  System  Outputs 

d.  Basics  IV  -  Understanding  and  Interpreting  Training 
Characteristics  Estimates 

e.  Basics  V  -  Understanding  and  Interpreting  High 
Driver,  Design  Problems,  and  Design  Principles  Data 

f.  Advanced  Topics  I  -  Sensitivity  Analysis:  Why  and 
How 

g.  Advanced  Topics  II  -  How  the  Training  Estimation 
System  Works 

h.  Advanced  Topics  III  -  Adjusting  Composite  Comparison 
Cases 

i.  Advanced  Topics  IV  -  Adjusting  Training  Character¬ 
istics  Estimates 

j.  Advanced  Topics  V  -  The  Role  of  Subject  Matter 
Experts  and  How  to  Use  SME  Information 

The  first  five  modules  are  designed  to  enable  the  first¬ 
time  user  to  utilize  the  Product  to  derive  basic  training 
charac ter i st ics  estimates.  The  last  five  deal  with 
advanced  topics  for  more  experienced  or  interested  users, 
or  those  who  need  to  use  the  more  advanced  capabilities  of 
the  Product. 

2.  Guided  Practice  and  Worked  Examples:  Much  of  the  training 
provided  by  the  ET  component  will  consist  of  hands-on 
exercises  with  extensive  guidance  for  the  user.  Exercises 
will  concentrate  on  accomplishing  specific  steps  of  using 
the  Product,  and  will  contain  error  diagnostics  (comparing 
the  user's  performance  with  that  of  an  idealized  Product 
Four  user)  to  enable  feedback  and  learning  from  errors. 

3.  A  Balanced  Mix  of  Knowledge  and  Hands-on  Training:  Some 
users  will  be  uninterested  in  "how  the  Product  works,"  and 
will  wish  to  emphasize  practical  capabilities.  Others  will 
develop  an  interest  in  how  the  Product  does  what  it  does  to 
produce  its  outputs.  The  content  and  structure  of  training 
will  accommodate  both  extremes,  as  well  as  many 
intermediate  points  on  the  "theory-practice"  continuum. 
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4.  Checkpoint  and  Resume :  We  recognize  that  the  Product's 
users  are  busy  people  with  many  demands  on  their  time.  We 
will  therefore  not  constrain  the  user  to  dedicate  time  to 
complete  even  a  single  module  of  training  at  one  session. 
Each  user's  progress  in  training  will  be  monitored  by  a 
control  feature  in  the  ET  component,  and  a  user  will  be 
able  to  suspend  training  at  any  point  and  resume  from  the 
same  point  at  a  later  time. 

5.  Basics  and  Advanced  Product  Use:  When  users  are  tasked  to 
develop  training  characteristics  estimates,  it  is  likely 
that  fast  response  will  be  particularly  important.  Thus, 
we  will  structure  training  to  enable  the  user  to  work  with 
basic  features  and  capabilities  of  the  Product  first.  If 
the  user  later  has  time  and  interest,  he  or  she  can  "add 
on"  training  in  how  to  work  with  such  features  as  the 
sensitivity  analysis  capability  or  adjusting  initial 
estimates  given  more  complete  data  or  access  to  SMEs. 

We  anticipate  that  the  ET  component  will  be  capable  of  enabling  a 
completely  naive  user  to  develop  basic,  "no-frills"  training 
characteristics  estimates  for  a  new  system  after  a  maximum  of  fc ur 
hours'  training  time,  given  straightforward  and  reasonably  complete 
system  and  subsystem  cha rac t e r i s t ics  data  for  the  contemplated  new 
system. 


Context-Sensitive  Help  and  Explanation  Facility 


This  facility  will  enable  the  user  to  request  help  and  explan¬ 
ations  at  any  time  the  user  is  actually  interacting  with  Product  Four. 
Context  sensitivity  of  this  feature  refers  to  the  fact  that  the  Product 
will  have  information  about  what  the  user  is  attempting  to  accomplish 
during  any  interaction.  Using  this  information,  the  Product  will 
provide  guidance  and  explanations  of  how  to  accomplish  the  particular 
function.  The  Product  will  also  be  able  to  present  information 
regarding  why  particular  inputs,  judgments,  etc.  are  needed  to 
accomplish  che  interaction.  Guidance  will  always  be  provided  when  the 
user  invokes  the  help  capability.  "Why"  information  will  only  be 
presented  at  the  explicit  request  of  the  user. 

The  user  interface  with  the  help  and  explanation  capability  will 
be  through  a  "hot-key"  approach,  with  one  function  key  (or  the 
equivalent)  always  set  aside  to  request  help.  If  the  information 
contained  in  the  help  or  explanation  requires  more  than  one  full 
display  screen  to  present,  the  user  will  utilize  the  normal  up  and  down 
cursor-control  or  scrolling  keys  to  move  forward  or  backward  through 
the  information  presented.  If  there  are  options,  choices,  or  responses 
associated  with  a  help  or  explanation  display,  the  user  will  be 
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presented  with  a  "pull-down"  menu  of  choices,  above  the  normal  display 
area  for  help  information.  Choices  will  be  made  by  moving  a  block 
cursor  to  the  desired  option  or  response  (using  the  left  and  right 
cursor  control  keys)  and  using  an  "enter"  or  execute  key  to  invoke  the 
choice  desired. 
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SECTION  EIGHT 


PRODUCT  FOUR  ACCEPTANCE 


Our  entire  approach  to  the  design  and  development  of  this  Product 
is  intended  to  foster  both  user  acceptance  of  Product  Four  and 
acceptance  of  its  outputs  by  those  who  use  them.  Specifically,  we  do 
this  in  the  following  ways: 

1.  We  identify  in  detail  what  the  Product  should  produce  as 
its  outputs,  for  each  potential  class  of  user.  This  works 
to  ensure  the  Product's  acceptability  by  providing  what  is 
genuinely  useful  to  the  users  of  the  outputs,  rather  than 
something  else. 

2.  We  involve  Product  users  in  the  design  and  testing  of  the 

Product.  This  ensures  that  we  will  be  able  to  design  out 

features  or  characterise ics  of  Product  operations  and  the 

user  interface  which  could  work  to  reduce  Product 
acceptance.  This  also  gives  some  "ownership"  of  the 
Product  to  at  least  some  of  the  people  who  will  use  it. 

3.  We  make  the  Product  very  easy  to  learn  and  use,  by 

designing  the  user  interface  for  maximum  comprehensibility 
and  support,  and  providing  Embedded  Training  for  users. 

This  tends  to  overcome  resistance  to  new  techniques  and 
reluctance  to  use  the  Product  which  would  exist  if  the 
Product  were  more  difficult  to  use. 

U.  We  provide  constantly  available,  context-sensitive  help  for 
Product  users.  This  helps  to  avoid  or  reduce  frustration 
that  might  develop  in  using  the  Product,  if  users  become 
"stuck"  or  at  a  loss  for  what  to  do  next  while  using  the 
Produc  t . 

5.  We  ensure  users  are  able  to  understand  what  data  are 
required  to  operate  the  Product,  through  using  data  models 
or  templates  to  guide  data  input.  This  also  helps  to  avoid 
frustration  in  using  the  Product. 

6.  We  provide  a  large  number  of  defaults,  even  in  input  data 
requirements,  so  that  users  are  able  to  work  with  the 
Product  even  in  cases  where  data  are  scant.  This  assures 
that  the  user  wil1  be  able  to  generate  at  least  "something" 
which  is  face-valid  as  a  result  of  using  the  Product. 

7.  We  consider  all  the  classes  of  users  in  the  design  of  this 
Product.  This  is  especially  important,  since  there  are 
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multiple  classes  of  users.  The  data  generated  by  using  the 
Product  have  been  designed  to  satisfy  user  requirements  to 
the  maximum  extent  possible,  given  users'  characteristics 
and  data  needs. 

8.  We  consider  the  context  in  which  the  Product  will  be  used. 
We  recognize  that  Product  Four  will  be  used  very  early  in 
the  acquisition  cycle,  to  both  develop  constraints  for 
MANPRINT  charac teristics  of  systems,  and  to  influence 
system  design  to  minimize  training.  This  is  reflected  in 
several  of  the  measures  discussed  above. 

In  addition  to  these  approaches ,  the  approaches  we  will  take  to 
institutionalize  Product  Four  (see  Section  Nine)  will  help  to  ensure 
acceptance  and  use  of  the  Product  and  its  outputs. 
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SECTION  NINE 


INSTITUTIONALIZING  PRODUCT  FOUR 


There  are  two  approaches  that  will  be  used  to  institutionalize  the 
Product.  One  is  "campaign"  actions  that  will  be  taken  to  institution¬ 
alize  the  Product.  The  second  is  to  build  into  the  Product  character¬ 
istics  that  will  help  insure  its  institutionalization  (inherent 
charac terist ics  ) . 


Campaign  Actions  to  Institutionalize  the  Product 


The  institutionalization  plan  includes  three  campaign  action 
items .  These  are : 

1.  Involving  potential  users  in  the  development  process; 

2.  securing  the  support  of  a  general  officer;  and 

3.  teaching  Product  use  to  combat  developers  within  their 
training  course. 

The  first  part  of  the  institutionalization  plan,  chronologically, 
will  be  getting  the  pctential  users  of  the  Product  involved  in  all 
stages  of  its  development.  The  potential  users  will  be  involved  in  the 
design  and  testing  stages  through  both  informal  commenting  and  more 
formal  pi  1 ot- te s t ing  .  A  frequent  dialogue  will  be  established  with  the 
more  cooperative  potential  users.  They  will  be  asked  for  input  on 
database  item  and  structures,  and  interface  design,  in  addition  to  the 
more  basic  procedures  of  the  Product. 

Potential  users  include  the  combat  deveiopeis  "in  the  trenches"  at 
each  of  the  schools.  In  addition,  their  superiors  up  to  the  typical 
heads  of  the  combat  developments  directorates  (Colonels)  will  be 
encouraged  to  become  involved  in  the  development  of  the  system. 
Similarly,  appropriate  personnel  at  headquarters  TRADOC  also  will  be 
involved.  Finally,  AMC  personnel  also  will  be  encouraged  to 
par t ic ipate . 

The  reasons  for  the  involvement  of  the  potential  users  are: 

1.  To  develop  a  critical  mass  of  positive  potential  users 

prior  to  the  availability  of  the  Product.  They  will  help 
institutionalize  the  Product  by  using  it  and  promoting  it. 
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2.  To  obtain  data  from  the  potential  urers  to  help  tailor  the 
Product  mere  to  their  needs  and  desires.  This  will  make  it 
a  more  attractive  Product  and  thus  help  insure  its 
institutionalization. 

3.  To  be  able  to  say  we  consulted  with  the  users  in  the 
development  of  the  Product  and  that  we  have  their  support 
and  approval. 

The  second  campaign  action  item  will  be  to  obtain  the  support  of  a 
general  officer.  This  will  be  accomplished  after  the  Product  is 
partially  developed,  so  that  its  rudiments  can  be  shown  and 
demonstrated  to  help  obtain  support.  Also,  the  interaction  and  support 
of  the  users  will  be  marshalled  to  gain  the  support  of  a  general 
officer.  In  addition,  briefings  and  white  papers  will  be  used  to  extol 
the  efficiency,  cost,  and  other  benefits  the  Product  will  yield. 

The  third  campaign  action  item  is  to  have  the  use  of  the  Product 
included  in  the  course  taught  to  combat  developers.  It  should  be 
possible  to  include  a  new  "piece"  in  the  course  specifically  devoted  to 
promoting  the  Product.  In  addition,  the  course  will  teach  the  combat 
developers  how  to  use  it  and  make  them  familiar  and  comfortable  with 
it.  All  of  these  additions  to  the  course  will  help  institutionalize 
the  Product. 


Inherent  Features  Fostering  Institutionalization 


There  are  six  specific  features  which  will  be  inherent  character¬ 
istics  of  the  Product  or  results  it  will  yield  which  will  foster  its 
institutionalization.  The  six  are: 


1 . 

Face  validity; 

2. 

Reduces  labor; 

3. 

Lowers  cost  of  combat  developer's 

process 

4. 

Lowers  cost  of  system  development 

» 

5. 

Produces  a  formatted  audit  trail; 

and 

6. 

Helps  develop  0&0  plans. 

Face  validity  will  come  through  interaction  with  the  users  during 
the  development  and  testing  of  the  Product.  Face  validity  will  go  a 
long  way  toward  institutionalizing  the  Product.  Face  validity  will 
partially  result  from  the  users'  feedback  during  the  design  and 
development  process.  Such  feedback  will  provide  guidance  for  the 
design  and  development  of  the  Product.  This  will  help  to  assure  Chat 
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users  have  some  "ownership"  in  the  Product,  and  that  it  will  meet  their 
needs  effectively. 

Obviously,  the  Product  will  result  in  a  reduction  of  the  amount  of 
labor  required  by  combat  developers  to  produce  training  estimates. 

This  feature  can  almost  in  and  of  itself  cause  it  to  be  institution¬ 
alized.  Many  system  development  efforts  have  resulted  in  elegant 
systems  that  were  never  used.  Often  this  was  because  the  systems 
required  the  users  to  do  more  than  they  did  before.  On  the  other  hand, 
everyone  appreciates  a  job  aid  that  actually  makes  their  job  easier, 
especially  if  it  is  because  they  have  to  do  less.  The  reduced  labor  on 
the  part  of  combat  developers  also  may  result  in  their  training 
estimation  process  costing  less. 

Similarly,  the  Product  will  lead  to  a  lower  cost  for  the  develop¬ 
ment  of  systems  than  was  previously  experienced.  This  will  be  the 
result  of  having  specific  training  constraints  available  before  system 
design,  alleviating  costly  modifications  that  would  have  to  occur  if 
training  requirements  of  the  new  system  were  excessive.  Costly 
redesign  efforts  prior  to  developmental  testing  and  any  retro-fitting 
after  operational  testing  may  be  avoided.  In  addition,  designing  the 
system  to  meet  specific  training  constraints  also  may  help  the  system 
avoid  some  product  improvements  after  fielding.  All  of  these  reduced 
cost  aspects  will  be  effective  in  obtaining  the  support  of  a  general 
officer. 

The  Product  will  result  in  a  formatted  audit  trail  which  will 
document  each  training  estimation  effort.  This  is  an  especially 
attractive  feature  of  the  Product,  because  the  present  training 
estimation  process  often  leads  to  unanswerable  questions  about 
decisions  and  outputs.  Part  of  the  reason  for  the  present  process 
resulting  in  unanswerable  questions  is  that  the  process  is  not 
procedu ra 1 i zed  and  thus  requires  more  of  an  audit  trail  than  a 
procedu ra 1 i zed  one.  Moreover,  the  present  process  provides  no  format 
or  easy  to  use  vehicle  for  generating  an  audit  trail. 

Finally,  the  Product  will  help  combat  developers  produce  an  0&0 
plan.  Again,  any  time  a  new  product  makes  someone's  job  easier,  the 
product  has  a  high  probability  of  being  used  and  thus  being 
institutionalized. 
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DUCT I ON 


The  Problem 

The  Army  is  heavily  involved  in  training.  In  fact,  under 
peacetime  conditions,  almost  all  Army  activities  can  be 
considered  as  training  of  one  kind  or  another.  These  training 
activities  range  from  resident  school  instruction — Basic  Combat 
Training  (BCT)  and  Advanced  Individual  Instruction  (AIT) — to  both 
formal  and  informal  on-the-job  training  (OJT)  conducted  in  a 
variety  of  locations.  Training  constitutes  a  multibillion  dollar 
activity  requiring  the  support  and  involvement  of  a  large  portion 
of  available  Army  personnel. 

One  aspect  of  the  Army's  training  activities  that  has 
received  considerable  attention  during  the  recent  era  of 
widespread  force  modernization  is  the  process  by  wnich  training 
requirements  and  constraints  for  emerging  systems  are  determined. 
Under  current  regulations,  Army  combat  developers  are  required  to 
determine  a  proposed  system's  training  needs  early-on  (i.e., 
during  concept  exploration)  and  use  the  resulting  information 
both  to:  (1)  constrain  materiel  design,  and  (2)  lay  the 
groundwork  for  deploying  the  eventual  training  system 
concurrently  with  the  materiel  system. 

In  spite  of  the  firm  regulatory  emphasis,  the  Army's  actual 
early-on  training  estimation  process  does  not  work  as  intended. 

In  most  system  development  efforts,  training  considerations  are 
not  used  to  constrain  equipment  design,  and  the  actual  process  of 
fielding  the  training  system  often  lags  far  behind  the  materiel 
acquisition  itself.  As  a  result,  systems  are  often  fielded 
without  a  fully-developed  training  support  package.  Moreover, 
the  training  that  is  eventually  provided  often  is  not  sufficient 
to  bring  trainees  to  the  performance  levels  necessary  for 
military  success. 

The  research  and  development  called  for  under  the  current 
effort  is  in  direct  response  to  the  problem  alluded  to  in  the 
previous  paragraph:  An  undisciplined  system  development  process 
in  which  weapons  are  designed  without  regard  for  personnel 
constraints — manpower  levels,  aptitude  distributions,  and 
training  capabilities.  When  this  undisciplined  process  is 
applied  to  system  after  system,  the  result  is  a  cumulative  demand 
for  skilled  personnel  that  is  beyond  the  capability  of  the  Army 
to  satisfy.  The  specific  focus  of  Product  4,  referred  to  in  the 
current  paper  as  the  Training  Estimation  Methodology  (TEM) ,  is 
the  process  by  which  training  requirements  and  constraints  are 
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identified  early-on  and  are  used  to  influence  both  later  training 
system  development  and  materiel  system  design  and  employment 
concepts . 


Object Ives 

The  Statement  of  Work  (SOW)  for  the  present  effort  describes 
the  TEH  as,  "...a  product  that  will  aid  TRADOC  and  DCSPER 
MANPRINT  personnel  in  providing  a  quick  estimate  of  the  types  of 
training  and  the  maximum  training  times  that  are  likely  to  apply 
prior  to  preliminary  design.  The  purpose  of  this  product  is  to 
determine  the  probable  parameters  of  training  so  that  hardware 
and  software  designers  know  the  training  limits  and  dc  ret  design 
for  unavailable  training."  The  SOW  goes  on  to  state  that,  "... 
this  training  estimate  is  made  prior  to  initial  system  design  by 
providing  limiting  information  to  hardware  and  software 
designers.  ...  the  purpose  is  not  to  identify  what  training  ought 
to  be,  but  to  tell  designers  what  it  is  likely  to  be  so  that 
hardware  and  software  designs  do  not  require  types  and  amounts  of 
training  which  are  very  unlikely  to  be  made  available." 

Our  interpretation  of  the  language  in  the  SOW  is  that  the 
sponsor  (i.e.,  the  Army  Research  Institute  [ARI])  desires  a 
training  estimation  methodology  with  the  following  general 
characteristics: 

•  It  can  be  used  quickly 

•  It  must  be  usable  prior  to  initial  design 

•  It  must  identify  the  constraints  boundary  for  a  proposed 
materiel  system's  training  system 

•  It  must  identify  what  training  is  likely  to  be,  rather 
than  what  it  ought  to  be 

•  It  must  provide  results  usable  by  personnel  developing 
training  programs  as  well  as  hardware  and  software 
interface  designers 

In  other  words,  the  objective  of  the  TEM  is  to  provide  a 
rapid,  early-on  assessment  of  the  most  probable  architecture  of  a 
proposed  materiel  system's  associated  training  system. 

Information  regarding  the  most  probable  training  system 
architecture,  or  training  concept,  if  you  like,  is  to  be  used  as: 
(1)  a  basis  for  follow-on  training  development,  and  (2)  to 
influence  the  design  of  the  materiel  system  itself.  The  SOW 
makes  the  point  that  a  primary  objective  of  the  TEM  is  to 
constrain  system  design  by  providing  information  regarding 
training  parameters  so  that  system  designers  do  not  design  for 
unavailable  training. 
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A  Definition  of  "Most  Likely  Training'-1 

Our  experiences  concerning  the  way  in  which  the  Army 
develops  training  have  indicated  that  "most  likely  training,"  or 
that  training  which  emerges  from  the  Army's  de  facto  training 
design  and  development  process,  is  most  often  a  function  of  the 
following  influences: 

•  The  predecessor  (or  a  referent)  system's  training  program 

•  Documented  deficiencies  in  the  predecessor  training 
program 

•  Unique  or  new  training  requirements  imposed  by  a  new 
materiel  system 

•  The  proponent  school 't;  "world  view"  regarding  training 

•  The  available  and  projected  resources  base 

•  Advances  in  training  technology 

The  Army  rarely  duplicates  a  predecessor  or  referent 
training  system.  Rather,  the  training  system  for  a  new  materiel 
item  usually  represents  a  complex  compromise  between  training 
needs,  various  types  of  constraints,  institutional  biases,  past 
practices,  and  technological  advances  (i.e.,  what's  "hot"  now, 
training-wise) .  (Note:  The  current  emphasis  on  embedded 
training  is  a  good  example  of  the  "what's  hot  now"  phenomenon) . 
Most  new  training  systems  can  thus  be  characterized  as  consisting 
of  training  program  modifications  fit  within  explicit 
constraints.  The  term  "most  likely  training"  will  thus  be 
defined,  for  purposes  of  the  current  discussion,  as  a  training 
system  modification  (usually  an  upgrade)  accomplished  within 
constraints.  It  should  be  further  noted  that  such  training 
system  modifications  will  be  achieved  in  an  evolutionary  as 
opposed  to  revolutionary  fashion.  It  is  also  recognized  that  a 
system  developed  in  such  an  evolutionary  manner  will  rarely  be 
characterized  by  training  professionals  as  "ideal"  or  "optimal." 


Approach 

In  view  of  the  above  definition,  the  problem  of  developing 
the  TEM  is  thus  reduced  to  two  major  tasks.  The  first  is  to 
define  a  practical  methodology  that  will  aid  Army  analysts  in 
predicting  or  defining  the  scope  of  training  system  evolutionary 
modifications  that  can  be  fit  within  existing  or  projected 
constraints.  The  second  is  to  ensure  that  the  methodology  is 
institutionalized  and  used  by  analysts  early-on  in  the  system 
acquisition  process.  As  noted  in  the  SOW  for  the  current  effort, 
ensuring  the  use  and  acceptance  of  a  methodology  is  one  of  the 
critical  elements  of  a  solution  to  any  of  the  MPT  estimation 
problems . 
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We  believe  that  the  user  acceptance  problem  is  the  most 
difficult  of  the  two  tasks  which  must  be  accomplished  in 
development  of  a  viable  TEM  (Product  4).  Review  of  the  military 
training  literature  indicates  that  a  number  of  training 
development  aids  have  been  developed  but  none  have  gained 
widespread  use  or  acceptance  within  the  Army.  If  previous 
attempts  to  aid  analysts  in  determining  training  requirements 
have  failed  to  gain  acceptance,  the  problem  of  gaining  acceptance 
of  a  methodology  to  predict  the  "most  likely  training”  will  be 
substantial.  This  assertion  is  based  on  the  simple  fact  that 
analyses  to  determine  training  requirements  are  likely  to  be 
perceived  as  a  more  critical  need  by  Army  training  analysts  than 
prediction  of  "most  likely  training.”  Since  decision  aids  to 
support  the  former  need  have  failed  to  gain  widespread  Army 
acceptance,  the  design  of  a  methodology  to  predict  "most  likely 
training"  must  be  highly  sensitive  to  user  acceptance  if  it  is  to 
have  any  chance  of  success. 

The  key  to  designing  a  TEM  which  will  be  used  by  analysts 
throughout  the  Army  is  to  understand  the  problems  they  must  solve 
and  the  environment  in  which  they  work.  The  Army,  specifically 
the  Training  and  Doctrine  Comm, and  (TRADOC)  ,  has  a  very  elaborate 
and  well-defined  formal  process  for  training  design  and 
development.  This  system  has  the  formal  charter  for  all  early-on 
training  design  and  development  activities.  If  early-on  training 
design  or  development  results  are  to  have  official  sanction,  they 
must  flow  from  or  at  least  receive  the  imprimatur  of  this  formal 
system.  Hence,  the  TEM  must  conform  to  the  general  requirements 
of  the  formal  TRADOC  system  if  it  is  to  have  any  chance  of  being 
accepted  into  the  stable  of  sanctioned  training  analysis  tools. 

In  fact,  it  can  be  stated  categorically  that  the  primary  reason 
that  previous  efforts  to  institutionalize  TEM-like  methodologies 
(and  there  have  been  several)  have  failed  is  that  they  have  not 
been  accepted  by  the  formal  system.  With  this  fact  in  mind,  one 
of  our  primary,  non-technical  objectives  for  the  TEM  is  that  it 
He  orthodox;  the  methodology  must  be  perceived  as  being  in 
complete  conformity  with  TRADOC's  formal  regulatory  apparatus  and 
possess  enough  flexibility  to  accommodate  procedural  variations 
at  each  of  the  local  combat  development  centers. 

The  focus  on  user  acceptance  and  institutionalization  of  the 
TEM  is  the  :,high  driver"  guiding  the  development  of  the  concept 
for  Product  4  presented  in  the  current  paper.  The  tv~  cth : r 
problems  noted  in  the  SOW  (obtaining  data  to  conduct  the  required 
analysis  and  development  of  information  provided  by  the  MPT 
estimation  product)  have  to  be  solved  within  the  design 
constraints  dictated  by  the  user  environment.  The  MPT  estimation 
tools  (like  any  system  to  be  analyzed  using  MPT  estimation  tools) 
must  be  designed  considering  the  "performance  of  the  soldiers  who 
operate,  maintain,  and  support  their  hardware  and  software." 
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Organization  of  the  Concept  Paper 


The  remainder  of  the  concept  paper  is  organized  to  provide 
the  information  required  by  the  SOW  and  to  present  a  detailed 
discussion  of  the  approach  the  current  authors  believe  is 
required  to  develop  a  viable  Product  4.  Section  2  of  the  paper 
presents  a  detailed  analysis  of  the  regulatory  environment  in 
which  the  users  of  Product  4  must  operate.  To  gain  acceptance, 
Product  4  must  contribute  to  the  accomplishment  of  tasks  existing 
in  this  environment.  Included  in  this  section  of  the  concept 
paper  is  a  description  of  the  projected  users  of  Product  4. 
Section  2  ends  with  a  discussion  of  the  design  constraints  for 
Product  4  dictated  by  the  user  environment. 

Section  3  of  the  concept  paper  discusses  the  structure  and 
function  of  the  TEM.  Included  in  this  section  is  a  description 
of  the  data  required  to  exercise  the  TEM  and  a  discussion  of  the 
output  generated  by  the  methodology.  Other  issues  discussed  in 
Section  3  include  how  the  TEM  will  be  developed,  how  it  will 
train  its  users,  and  estimates  of  the  person-hours  required  to 
use  the  TEM  to  identify  the  "most  likely  training"  for  a  system 
in  the  concept  development  phase. 


THE  TEM  USER  ENVIRONMENT 


Introduction 

The  overall  context  for  the  application  of  the  suite  of 
early-on  Manpower,  Personnel,  and  Training  (MPT)  estimation  tools 
called  for  under  the  SOW  is  the  Army's  Weapons  System  Acquisition 
Process  (WSAP) .  A  myriad  of  regulations  and  documents  (e.g.,  AR 
70-1,  DA  PAM  11-25,  TRADOC/AMC  Pamphlet  70-2,  etc.)  describe  the 
process  by  which  the  Army  acquires  materiel  systems  and  addresses 
human  performance  issues  as  part  of  the  developmental  process. 

For  purposes  of  the  current  effort,  the  Army's  systems 
development  process  can  be  viewed  as  proceeding  much  as  depicted 
in  Fioure  1.  Periodic  Mission  Area  Analyses  (MAAs)  are  used  to 
identify  threat-related  deficiencies  on  the  part  of  the  current 
force.  In  response  to  mission  area  deficiencies,  an  evaluation 
is  made  of  ways  in  which  a  capabilities  gap  can  be  closed. 

Issues  that  are  addressed  first  in  this  regard  include:  (1)  an 
evaluation  of  potential  dcctrir.cl  or  tactical  adjustments,  (2) 
manpower  or  personnel  solutions,  and  (3)  training  solutions.  If 
a  deficiency  cannot  be  redressed  through  doctrinal,  tactical,  or 
MPT  means,  then  a  materiel  solution  might  be  sought. 

If  a  materiel  solution  to  mission  area  deficiencies  is 
decided  upon,  the  initiating  documents  for  the  developmental 
effort  are  the  Justification  for  Major  System  New  Start  (JMSNS) 
and  the  Organizational  and  Operational  (O&O)  Plan.  In  theory,  at 
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Figure  1.  Early  training  activities 


least,  both  the  JMSNS  and  the  0«0  Plan  are  to  reflect  MPT 
constraints.  At  present,  however,  MPT  issues  are  given  only 
cursory  treatment  in  these  documents. 

TRADOC/AMC  Pamphlet  70-2,  Materiel  Acquisition  Handbook, 
describes  policy,  procedures,  and  responsibilities  for  acquiring 
materiel  systems.  Recently  (August  1986),  TRADOC  released 
detailed  guidance  concerning  the  consideration  of  the  six 
MANPRINT  (Manpower  and  Personnel  Integration)  domains  during  the 
development  of  all  materiel  items.  This  guidance  is  organized 
around  the  following  five  topics: 

1.  System  MANPRINT  Management  Plan  (SMMP) 

2.  Manpower  and  Personnel 

3.  Training 

4 .  Human  Factors  Engineering 

5.  System  Safety  and  Health  Hazard  Assessment 

Recruirements  for  Earlv-on  Training  Analyses 

A  review  of  Section  3  of  the  TRADOC  guidance  indicates  that 
training  activities  in  support  of  the  materiel  acquisition 
process  are  to  begin  prior  to  Program  Initiation.  Figure  1  shows 
the  general  flow  of  training  activities  beginning  with  the 
identification  of  a  need  to  develop  a  materiel  item  through 
Milestone  I.  The  Directorate  of  Training  and  Doctrine  (DOTD)  at 
the  proponent  school  is,  in  almost  all  cases,  responsible  for  the 
conduct  of  these  activities.  Also  shown  in  Figure  1  are  the 
major  analyses  to  support  MANPRINT.  These  analyses,  though  not 
specifically  training  related,  can  provide  information  which  is 
useful  in  developing  training  concepts  and  in  determining 
potential  problem  areas.  The  primary  objectives  of  these 
activities  can  be  characterized  as  follows: 

•  Develop  training  constraints  within  which  the  Army  must 
tailor  new  training  required  as  a  part  of  the  developed 
materiel  item. 

•  Develop  training  concepts  that  can  potentially  satisfy 
the  training  requirements  associated  with  the  developed 
materiel  item. 

•  Determine  the  most  cost-effective  approach  to  development 
and  delivery  of  required  training. 

Each  activity  is  addressed  in  more  detail  in  the  paragraphs 
to  follow. 

The  responsibility  for  developing  training  constraints  is 
vested  with  the  proponent  DOTD  -  New  Systems  Training  Office 
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(NSTO)  .  Constraints  should  be  developed  from  guidance,  lessons 
learned,  MAAs,  Post-Fielding  Training  Effectiveness  Analyses 
(TEA  ) ,  or  an  Early  Comparability  Analysis  (ECA) .  A  list  of 
constraints  should  be  developed  and  included  in  the  0&0  Plan  and 
the  SMMP.  These  constraints  should  also  be  considered  as  the 
Individual  and  Collective  Training  Plan  (ICTP)  is  developed. 
Specific  training  constraints  should  include  information 
concerning  the  treatment  of  high-driver  tasks,  time-to-train 
limits,  instructor  limits,  and  the  like. 

Once  training  constraints  are  established,  there  is  no 
formal  provision  for  revising  them.  However,  because  training 
constraints  are  included  in  the  0&0  Plan,  there  is  an  opportunity 
to  review  training  constraints  prior  to  Milestone  I.  Training 
constraints  may  also  be  included  as  part  of  the  Required 
Operational  Capability  (ROC) .  When  this  is  the  case,  a  review  of 
the  ROC  may  result  in  specific  program  guidance  addressing  the 
adequacy,  accuracy,  or  completeness  of  training  constraints. 

The  proponent  DOTD  -  NSTO  also  has  the  responsibility  for 
developing  the  training  concept  for  the  emerging  materiel  item. 
According  to  TRADOC  Regulation  350-4,  this  process  involves  the 
identification  of  generic  training  tasks,  course  requirements, 
training  sequences  and  locations,  and  an  initial  estimate  of 
training  equipment  and  devices  that  will  be  required. 

Alternative  training  approaches  and  concepts  should  be  included 
for  later  cost  evaluation.  This  activity  is  expected  to  be 
iterative  in  nature,  particularly  if  the  doctrine  associated  with 
a  new  system  is  still  under  development. 

Information  developed  in  support  of  the  training  concept 
should  then  be  used  as  a  basis  for  the  draft  ROC  training 
paragraphs  and  annexes.  Further  refinement  of  the  training 
concept  will  also  occur  in  the  development  of  the  ICTP.  Although 
not  currently  specified,  training  concept  development  could  be 
very  useful  in  identifying  potential  training  problem  areas.  It 
also  presents  the  proponent  with  an  opportunity  to  incorporate 
new  training  technologies  which  might  reduce  training  cost, 
manpower  requirements,  or  time.  In  general,  the  training  concept 
is  expected  to  undergo  continual  refinement  and  progressive 
resolution  over  the  course  of  the  acquisition. 

The  ICTP  is  the  capstone  document  for  all  training  concepts. 
According  to  TRADOC  Regulation  351-9,  the  "ICTP  must  present  a 
systematic,  workable  strategy  for  training  from  initial 
qualification  through  sustainment  of  that  proficiency  ...  and  for 
training  required  to  qualify  personnel  on  the  higher-level  tasks 
associated  with  the  equipment  as  they  advance  in  grade."  A  major 
pitfall  to  be  avoided  when  developing  the  training  concept 
presented  in  the  ICTP  is  to  fix  on  concrete  training  solutions 
(i.e.,  explicit  alternatives)  too  early  in  the  acquisition 
process. 
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Cost-effective  training  approaches  are  determined  within  the 
context  of  the  TEA  system.  The  TEA  process  is  divided  into  two 
broad  categories:  (1)  developmental  TEAs  (DTEAs) ,  and  (2)  post- 
fielding  TEAs.  The  post-fielding  TEA  is  not  applicable  to  the 
initial  phases  of  the  materiel  acquisition  process.  As  the  name 
implies,  it  is  conducted  after  th_:  materiel  item  is  provided  to 
units.  The  DTEA  consists  of  three  elements:  (1)  a  Preliminary 
Training  Effectiveness  Analysis  (PTEA) ,  (2)  the  Cost  and  Training 
Effectiveness  Analysis  (CTEA) ,  and  (3)  the  Training  Development 
Study  (TDS) . 

The  Preliminary  Training  Effectiveness  Analysis  (PTEA)  is 
conducted  to  facilitate  the  development  of  training  concepts  and 
strategies  (TRADOC  Regulation  350-4).  In  general,  the  PTEA 
should  address  the  target  audience,  the  identification  of  skills 
to  be  trained,  and  a  first  cut  at  the  "when,  where,  and  how"  of 
training.  The  PTEA  is  intended  to  support  decisions  at  the  end 
of  the  concept  exploration  phase,  Milestone  I  of  the  WSAP.  This 
information  also  provides  a  basis  for  developing  the  Outline  ICTP 
(OICTP) . 

The  Cost  and  Training  Effectiveness  Analysis  (CTEA)  is 
characterized  as  a  detailed  analysis  of  the  comparative 
effectiveness  and  cost  of  the  training  alternatives  developed  to 
support  a  new  materiel  item.  By  regulation,  the  CTEA  is  intended 
to:  (1)  establish  the  proficiency  attained  by  different  training 
alternatives,  (2)  determine  their  cost,  (3)  perform  a  cost- 
effectiveness  trade-off,  (4)  recommend  a  preferred  training 
alternative,  and  (5)  verify  or  amend  the  training  concept  and 
strategies  presented  in  the  OICTP.  The  CTEA  will  support 
decisions  at  the  end  of  the  demonstration  and  validation  phase, 
Milestone  II  of  the  WSAP. 

The  purpose  of  the  Training  Development  Study  (TDS)  is  to 
evaluate  the  efficiency  of  a  training  device  or  simulator  prior 
to  full-scale  production.  According  to  TRADOC  Regulation  350-4, 
a  TDS  "must  analyze  the  cost  and  effectiveness  of  a  simulator  and 
how  it  will  support  training  for  a  fielded  system  or  non-system 
training  program."  It  can  be  performed  as  part  of  a  PTEA,  CTEA, 
or  CTEA  update. 


Summary  of  the  TRADOC  W\SP  Training  Analysis  Requirements 

In  summary,  the  formal  TRADOC  training  design  and 
development  system  described  in  the  previous  paragraphs  defines  a 
two-level  process  for  early-on  training  requirements  definition. 
The  first  level  of  the  process  is  embodied  in  the  PTEA.  This 
analysis  is  to  be  performed  prior  to  Milestone  I  when  only  system 
concepts  exist.  Again,  the  objective  of  the  PTEA  is  the 
performance  of  preliminary  analyses  and  design  activities 
directed  at  training  concept  development.  By  regulation,  the 
PTEA  is  to  examine:  (1)  training  constraints;  (2)  potential 
training  problems;  and  (3)  the  definitions  of  a  training  system 


concept  addressing  (a)  what  is  to  be  trained,  (b)  who  is  to  be 
trained,  and  (c)  a  first  cut  at  the  "when,  where,  and  how"  of 
training.  An  initial  TDS  addressing  training  device  concept 
development  can  also  be  performed  as  part  of  the  PTEA.  Output 
from  the  PTEA  is  used  to  structure  the  OICTP. 

Level  two  of  the  formal  process  training  development  process 
is  the  CTEA.  The  CTEA  is  performed  prior  to  Milestone  II  during 
demonstration  and  validation  when  materiel  system  prototypes 
exist.  It  consists  of  detailed  analysis  and  design  activities 
directed  at  individual  course  (i.e.,  MOS-specific  training) 
development,  including  a  detailed  consideration  of  training 
device  requirements.  Conceptually,  the  CTEA  is  performed  within 
the  context  of  the  system-wide  training  architecture  defined  at 
the  PTEA  stage.  Output  from  the  CTEA  provides  the  structure  for 
individual  course  development  and  is  used  to  transform  the  OICTP 
into  the  ICTP. 


The  Realities  of  Earlv-on  Training  Analysis 

The  description  provided  in  the  previous  paragraphs  is  of 
the  manner  in  which  the  Army's  early-on  training  analysis  and 
design  process  is  supposed  to  work,  not  how  it  actually  is 
carried  out.  Needless  to  say,  the  consideration  of  training 
during  system  development  does  not  actually  proceed  as  described 
in  Figure  1.  Most  of  the  time,  training  issues  are  not  addressed 
explicitly  prior  to  Program  Initiation.  As  a  result,  program 
initiating  documents  typically  do  not  pay  more  than  lip  service 
to  training  issues  (e.g.,  a  token  mandate  not  to  require  more 
training  assets  than  the  predecessor).  As  an  added  consequence, 
training  constraints-related  information  is  usually  not  available 
to  system  designers  early-on. 

Between  Program  Initiation  and  Milestone  I,  the  training- 
related  actions  described  in  Figure  1  are  usually  performed  in  a 
token  fashion  or  are  waived.  As  a  result,  the  system's  training 
concept  is  not  addressed  explicitly.  Consequently,  the  system- 
level  concept  is  defaulted  away,  and  any  ensuing  analyses  tend  to 
focus  immediately  on  the  "nuts  and  bolts"  of  training  for 
individual  courses.  Individual  course  development  then  proceeds 
in  isolation  with  no  training-system-level  coordination  or 
synergy.  It  has  been  our  observation  that  the  default  treatment 
of  the  training  concept  brought  on  by  not  performing  a 
comprehensive  PTEA  early-on  is  one  of  the  primary  reasons  for  the 
poor  track  record  attributed  to  the  Army's  formal  training  design 
and  development  process. 

At  a  more  specific  level,  some  of  the  symptoms  of  the  lack 
of  an  early-on,  system-wide  focus  for  training  include  the 
following: 

1.  There  is  considerable  confusion  over  the  split  between 
training  concept  development  and  the  development  of  training 
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alternatives  within  that  conceptual  framework.  In  other  words, 
training  development  personnel  tend  not  to  understand  the 
rationale  for  the  PTEA/CTEA  split;  the  PTEA  and  the  CTEA  are 
viewed  as  addressing  the  same  issue,  which  is  not  the  case  at 
all.  As  discussed  above,  the  result  is  that  the  all-important 
training  concept  development  step  often  is  not  performed. 

2.  Early-on  training  design  and  development  is  late  and  out 
of  synchrony  with  the  rest  of  the  system  development  process. 
Typically,  training  issues  are  not  considered  explicitly  within 
DOTD  -  NS TO  until  after  Milestone  I  (i.e.,  not  until  the  CTEA  is 
initiated).  By  that  time,  however,  the  training  "metal  is  bent," 
quite  literally,  in  the  form  of  very  explicit  training  device 
concepts.  The  training  community  is  then  relegated  to  doing  what 
has  been  termed  "Procrustean  training  analyses"  to  justify  a 
proposed  training  device  concept  that  may  or  may  not  be  warranted 
learning-wise  or  even  cost-effective.  Procrustean  training 
analyses  are  training  analysis  and  design  activities  performed  to 
accommodate  an  existing  training  device  concept.  Under  such  a 
procedure,  training  devices  are  proposed  first,  and  the  remainder 
of  the  training  program  then  constructed  around  the  major 
devices.  The  term  is  derived  from  the  mythical  Greek  bandit 
Procrustes  and  his  infamous  bed. 

3.  There  are  no  standardized  procedures  for  the  conduct  of 
PTEA  or  CTEA.  This  observation  is  particularly  true  for  PTEA. 
Various  CTEA  methodologies  and  supporting  tools  exist  (see  Adams 
&  Rayhawk,  1986) ,  but  the  Army  has  not  yet  determined  a  standard 
version.  As  a  result,  a  "Tower  of  Babel"  syndrome  exists  within 
the  TEA  world.  TRADOC  schools  both  conduct  in-house  and  contract 
for  various  TEAs.  Given  the  divergence  of  methodologies, 
however,  there  is  no  assurance  that  any  two  analyses  will  stem 
from  the  same  conceptual  reference  point. 


The  TEM  Concept  in  the  TEA  Environment 

Based  upon  a  review  of  the  way  in  which  the  Army's  TEA 
system  (the  formal  basis  for  Army  training  design  and 
development)  is  implemented,  the  most  obvious  deficiency  is  the 
PTEA.  As  noted  previously,  the  PTEA  is  to  be  performed  during 
concept  exploration  for  the  expressed  purpose  of;  (1)  determining 
constraints,  (2)  identifying  problem  areas,  and  (3)  formulating  a 
system-wide  training  concept.  A  review  of  the  objectives  of 
Product  4  (i.e.,  the  TEM)  as  outlined  in  the  SOW  indicates 
similar  objectives.  Given  this  similarity  in  objectives,  we  have 
determined  that  the  most  effective  way  of  meeting  the 
requirements  of  the  current  effort  and  producing  a  tool  that  will 
gain  acceptance  and  be  used  in  the  Army  is  to  develop  a 
methodology  to  support  the  conduct  of  PTEA. 

The  decision  to  develop  a  TEM  designed  to  aid  in  the  conduct 
of  the  PTEA  allows  the  identification  of  the  users  of  the  product 
as  well  as  specification  of  certain  attributes  the  TEM  must 
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possess.  The  paragraphs  which  follow  briefly  describe  the  user 
audience  and  the  design  characteristics  which  the  TEM  oust 
possess  to  aid  in  the  PTEA  process. 


The  TEM  Users 

The  conduct  of  the  PTEA  is  the  responsibility  of  the 
Directorate  of  Training  and  Doctrine  -  Hew  Systems  Training 
Office  (DOTD-NSTO)  of  the  proponent  TRADOC  Combat  Development 
Center  for  the  new  system.  Thus,  the  staff  typically  found  in 
the  DOTD-NSTO  can  be  assumed  to  represent  the  users  of  the  TEM. 
Experience  indicates  that  the  composition  of  the  staff  working  in 
the  DOTD-NSTO  varies  from  one  school  to  the  next,  but  typically 
includes  Army  majors  and  captains,  one  or  two  civilian  training 
analysts,  and  Subject  Matter  Experts  (SMEs)  who  are  often  senior 
enlisted  personnel.  Thus,  the  "team”  using  the  TEM  may  be 
expected  to  have  a  reasonable  familiarity  with  the  training 
analysis  process  (as  practiced  at  that  proponent  school).  The 
team  will  also  include,  or  at  least  have  access  to,  SMEs  with 
knowledge  relevant  to  the  predecessor  systems  for  the  new  system 
under  analysis. 

The  team  using  the  TEM  will  also  likely  contain  several 
members  who  are  capable  of  using  a  computer  terminal  or  micro¬ 
computer.  While  the  TEM  users  can  be  assumed  to  have  a  minimum 
degree  of  familiarity  with  using  micro-computers  or  terminals, 
one  cannot  assume  familiarity  with  any  particular  software 
system. 

Our  previous  experience  in  working  with  individuals  in  the 
DOTDs  for  several  proponent  combat  development  centers  also 
suggests  certain  biases  regarding  analytic  tools.  First,  the 
users  tend  to  be  reluctant  to  use  any  tool  that  is  not  fully 
understandable  to  them.  In  particular,  these  individuals  are 
particularly  sensitive  to  assumptions  made  in  any  transformation 
of  data  and  the  rules  or  algorithms  used  during  analyses. 
Furthermore,  the  users  tend  to  prefer  tools  that  allow  them  to 
generate  and  compare  alternatives  over  time  as  new  information 
comes  available. 

Finally,  it  is  safe  to  assume  that  the  TEM  users  will 
operate  under  fairly  severe  time  constraints.  Combat  development 
centers  typically  carry  a  heavy  workload  with  a  minimum  of 
personnel.  In  recent  years  these  centers  have  experienced 
increased  taskings  with  no  increase,  and  in  many  cases,  with  a 
reduction  in  personnel.  Thus,  the  TEM  user  will  need  a 
methodology  which  is  as  time  efficient  as  possible. 


Objectives  for  the  TEM 

The  decision  to  develop  a  TEM  which  supports  the  PTEA 
process  allows  us  to  identify  several  specific  issues  which  must 
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be  addressed  to  gain  user  acceptance  of  the  TEM.  In  pursuing  the 
development  of  a  comprehensive  PTEA  methodology,  the  following 
objectives  will  be  addressed: 

1.  Orthodoxy.  The  TEM  must  meet  TRADOC's  formal 
requirements  for  a  PTEA  methodology  as  outlined  in  TRADOC 
Regulation  350-4  and  other  documents  describing  the  systems 
approach  to  training  (SAT)  design. 

2.  Sufficiency.  At  a  minimum,  the  TEM  must  address  the 
functional  requirements  for  PTEA  outlined  by  TRADOC  and  must  meet 
the  requirements  of  the  current  effort  as  cited  in  the  SOW. 

3.  Standardization  without  over  formalization.  The  TEM 
resulting  from  the  present  effort  must  meet  the  goal  of 
standardization  without  over  formalization.  It  is  our  objective 
to  provide  the  Army  with  a  standardized  tool  to  support  the 
conduct  of  PTEA.  However,  we  do  not  want  to  over  formalize  the 
resulting  procedure  to  the  point  where  it  becomes  unnecessarily 
cumbersome  or  cannot  accommodate  local  procedural  variations. 

Over  formalization  is  one  of  the  primary  reasons  that  many 
previous  TEM-like  tools  have  not  achieved  their  design  potential. 

4.  User  Friendly.  The  procedure  must  be  readily  usable  by 
personnel  representative  of  DOTD-NSTO  job  incumbents. 

5.  Economical.  The  procedure  must  not  require  more 
resources — people,  money,  technical  support  (computers,  software, 
etc.) — than  is  currently  available  within  a  typical  DOTD 
environment  at  a  combat  development  center. 

6.  Synchrony.  The  TEM  must  assist  in  fostering  a  synchrony 
between  training-related  acquisition  events  and  materiel-oriented 
aspects  of  the  system  development  process. 


Overview  of  the  Structure  of  the  TEM 

At  the  present  time,  we  conceive  of  the  TEM  as  a  three-part 
methodology,  with  each  part  being  referred  to  as  a  "module."  The 
TEM  is  a  computer  assisted  analytic  methodology.  It  is  not  a 
fully  automated  model.  That  is,  the  automated  portions  of  the 
TEM  guide  the  user  through  an  analytic  process  including 
prompting  the  user  for  data,  identifying;  potential  data  sources, 
constructing  data  bases,  performing  analyses,  and  generating 
reports.  The  wide  variety  of  systems  for  which  the  TEM  might  be 
applied,  variation  in  the  amount  and  quality  of  data  available 
for  different  systems,  and  a  variety  of  other  factors  require 
that  the  user  play  an  active  role  in  the  TEM.  The  three  modules 
comprising  the  TEM  are  listed  and  described  as  follows: 

e  Training  Constraints 

e  Training  Analysis 
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•  Feedback  and  Interface 

Briefly  stated,  the  objective  of  the  Training  Constraints 
Module  is  to  assist  training  developers  in  identifying  relevant 
constraints  within  which  an  emerging  training  system  must  fit. 
Constraints  identified  include  resources,  technology,  world  view, 
training  program  performance  history  and  potential,  and  other 
similar  information. 

As  the  name  implies,  the  Training  Analysis  Module  is 
intended  to  lead  analysts  through  a  series  of  procedures  that 
support  the  definition  of  a  system-wide  training  concept.  The 
resulting  training  concept  will  address,  at  a  generic  level,  the 
"what,  who,  how,  when,  and  where"  of  training.  It  will  also 
enable  the  conduct  of  an  early-on  TDS  directed  at  the  definition 
of  a  range  of  training  device  concepts  that  conform  with  the 
overall  training  concept.  Training  concepts  emerging  from  the 
analysis  stage  will  all  be  achievable  within  the  limits 
identified  through  the  exercise  of  the  Constraints  Module. 

Finally,  the  Feedback  and  Interface  Module  is  intended  to 
serve  as  the  link  between  the  TEM  and  follow-on  training  design 
and  development  activities,  most  notably  the  CTEA.  This  module 
is  also  intended  to  serve  as  a  vehicle  for  structuring  training- 
related  feedback  to  concept  developers  and  materiel  designers, 
both  inside  and  outside  of  the  Army. 

Additional  details  concerning  the  TEM  concept  and  how  it 
will  fit  within  TRADOC's  formal  training  development  process  are 
provided  in  the  next  section,  Tne  Training  Estimation 
Methodology. 


E2-17 


THE  TRAINING  ESTIMATION  METHODOLOGY 


Overview  of  the  TEM 


The  SOW  for  the  present  effort  indicates  that  Product  4, 
referred  to  herein  as  the  TEM,  is  intended  to  be  a  tool  that  will 
provide  combat  and  materiel  developers  with  an  outline  of  the 
training  "most  likely  to  be  made  available"  to  support  an 
emerging  system.  In  Section  1,  we  further  defined  the  term  "most 
likely  training"  to  mean  the  scope  and  nature  of  training  system 
modifications  that  can  be  accomplished  within  broad  classes  of 
constraints. 

Section  2  made  the  point  that  our  overall  concept  for  the 
TEM  is  to  develop  it  as  a  partial  guide  to  the  conduct  of  PTEAs. 
The  PTEA  is  an  essential  aspect  of  the  TRADOC  TEA  system,  but  one 
that  is  net  often  performed.  As  noted  earlier,  the  PTEA  is 
intended  to  address:  (1)  training  constraints,  (2)  potential 
training  problem  areas,  and  (3)  the  development  of  a  system-wide 
training  concept.  In  accord  with  this  view,  the  TEM  will  be 
developed  in  three  interrelated  parts,  or  modules,  listed  as 
follows : 


«  Training  Constraints 

•  Training  Analysis 

•  Feedback  and  Interface 


TEM  Structure 

A  graphic  description  of  the  overall  TEM  concept  is 
presented  as  Figure  2.  Each  of  the  component  modules  is  now 
described  in  an  overview  fashion.  More  detailed  descriptions  of 
the  modules  follow  under  their  own  headings. 

The  objective  of  the  Training  Constraints  Module  (TCM)  is  to 
assist  training  developers  in  identifying  and  characterizing 
relevant  training  constraints  for  an  emerging  materiel  system. 

As  shown  in  Figure  2,  input  to  the  TCM  consists  of  information 
concerning: 

•  Emerging  system  concepts 

•  Predecessor  or  referent  system (s)  and  their  training 
program (s) 

•  Training-related  Manpower  and  Personnel  constraints: 
Training  manpower  likely  to  be  available;  and  the 
personnel  footprint  of  the  predecessor  system,  often 
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Overview  of  the  training  estimation  methodology 
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referred  to  as  a  Target  Audience  Description  (i.e., 
output  from  Product  3  of  the  current  effort) 

Using  g  '.dance  provided  by  the  TCM,  users  are  led  through  a 
series  of  steps  leading  to  the  identification  of  the  following 
classes  of  training  constraints: 

1.  Resources.  A  description  of  the  training  resource  base 
likely  to  be  available  to  support  the  emerging  system. 

2.  Technology.  A  characterization  of  the  technological 
state  of  the  training  base  likely  to  be  available  to  support  the 
proposed  system. 

3.  Proponent's  World  View.  A  characterization  of  what  DoD, 
the  Army,  TRADOC,  and  the  proponent  school  think  about  training 
in  general  and  training  for  the  proposed  system. 

4.  Predecessor  or  Referent  (i.e.,  a  Baseline  Comparison 
System  [BCS])  System  Training.  The  "what,  where,  and  how"  of 
training  for  the  predecessor (s)  or  any  referent  system(s).  This 
will  consist  of  a  complete  characterization  of  any  predecessor  or 
referent  training  grcgram. 

5.  Predecessor  Training  System  Performance.  The  training 
capability  and  capacity  of  the  predecessor  or  referent  training 
system.  This  will  include  a  treatment  of  training  performance 
deficiencies  obtained  from  post-fielding  TEAs  or  other  sources. 

6.  Manpower  and  Personnel  Capabilities.  Numbers  of 
personnel  available  to  staff  the  training  system;  descriptions 
of  aptitude  and  abilities  for  target  audiences?  personnel-related 
high  drivers  on  predecessor  or  referent  systems. 

Output  from  the  TCM  is  structured  as  input  to  a  constraints 
data  base  that  is  available  to  support  early-on  training  analyses 
and  from  which  various  ad  hoc  reports  can  be  derived. 

The  Training  Analysis  Module  (TAM)  is  concerned  with  the 
development  of  a  system-wide  training  concept  within  the 
boundaries  of  the  constraints  identified  through  the  exercise  of 
the  Constraints  module.  Since  it  is  conceived  of  as  part  of  a 
PTEA  methodology,  the  TAM  must  address:  (1)  what  is  to  be 
trained,  (2)  who  is  to  be  trained  (i.e.,  potential  MOSs  and 
target  audience  descriptions) ,  and  (3)  the  "how,  where,  and  when" 
of  training.  As  opposed  to  the  CTEA  phase  of  the  TEA  process  and 
most  of  the  current  training  estimation  methodologies,  the  TAM 
will  function  at  the  concept  level.  Its  purpose  is  to  define  the 
concept-level  architecture  of  the  total  training  system  as 
opposed  to  the  detailed  structure  of  individual  courses  within 
that  system.  The  TAM  represents  the  analytical  "core"  of  the 
TEM. 
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Input  to  the  TAM  consists  of:  (1)  the  information  provided 
to  the  TCM,  (2)  information  derived  from  the  Constraints  module, 
and  (3)  information  from  Product  1  of  the  present  effort.  As 
noted  earlier,  most  likely  training  is  defined  as  training  system 
modifications  accomplished  within  limits.  The  TAM  will  indicate 
the  scope  of  beneficial  upgrades;  output  from  the  TCM  will 
delineate  the  boundaries  within  which  potential  training 
solutions  must  lie. 

TAM  output  consists  of  the  traditional  what,  where,  when, 
how,  order,  and  time  parameters  of  training.  However,  as  noted 
previously,  the  focus  of  the  TAM  is  on  the  architecture  of  the 
total  training  system  as  opposed  to  the  structure  of  individual 
courses.  TAM  analyses  are  thus  conducted  at  a  generic  as  opposed 
to  course-specific  level.  The  generic  focus  of  the  TAM  is  a  key 
to  making  it  a  simple  and  easy-to-use  procedure. 

The  final  component  of  the  TEM  is  the  Feedback  and  Interface 
Module  (FIM) .  The  FIM  is  intended  to  serve  as:  (1)  a  mechanism 
for  providing  training-related  feedback  to  system  stakeholders, 
and  (2)  an  interface  between  the  PTEA  phase  of  the  TEA  process 
and  follow-on  training  analysis  and  development  activities.  Our 
concept  for  the  FIM  is  that  it  will  function  as  a  query  and 
report  generation  capability  capable  of  producing  both  standard 
and  ad  hoc  reports.  Standard  reports  will  consist  of  training 
products  or  documentation  mandated  by  Department  of  the  Army  (DA) 
or  TRADOC  regulations.  For  these  reports,  both  the  content  and 
format  will  be  defined  as  part  of  FIM  development.  Ad  hoc 
reports  are  special  reports  of  a  "what  if"  nature  that  are  used 
to  support  acquisition  decision  making  or  later  training  design 
and  development  but  are  not  easily  anticipated.  The  FIM  data 
base  consisting  of  output  from  both  the  TAM  and  TCM  will  also  be 
used  to  initiate  the  more  detailed  training  design  and 
development  activities  that  occur  as  part  of  the  follow-on  CTEA 
process. 


Operation  of  the  TEM 

The  three  modules  of  the  TEM  are  designed  to  be  implemented 
sequentially.  The  user  should  first  exercise  the  training 
constraints  module  (TCM)  to  develop  the  various  resource 
constraint  data  bases.  The  training  analysis  module  (TAM)  is 
then  exercised  to  develop  a  description  of  recommended  training 
for  each  of  the  task  or  function  statements  produced  by  Product 
1.  In  the  final  step  in  the  TAM,  the  user  modifies  the 
recommended  training  to  fit  within  the  resource  constraints 
identified  in  the  TCM.  This  final  step  provides  the  TEM's 
prediction  of  the  "most  likely"  training  to  evolve  for  the 
emerging  materiel  system. 

After  the  TCM  and  TAM  have  been  exercised  to  develop  the 
required  data  bases  and  conduct  analyses,  the  Feedback  and 
Interface  Module  (FIM)  is  used  to  generate  a  variety  of  reports. 
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The  FIM  provides  a  variety  of  data  base  query  and  report 
generation  functions  which  will  allow  the  user  to  examine  and 
print  the  output  of  each  major  element  or  function  of  the  TCM  and 
TAM. 


Each  module  comprising  the  TEM  is  discussed  in  turn  in  the 
sections  to  follow. 


The_Training__Constraints  Module  (TCM) 


Objectives 

The  primary  objective  of  the  TCM  is  to  develop  structured 
data  bases  which  provide  the  user  with  information  on  resource 
and  constraint  factors  influencing  the  training  system  that 
emerges  for  the  materiel  system  of  interest.  The  data  bases  will 
serve  three  functions  in  the  TEM.  The  data  bases  provide: 

•  Information  on  training  constraints  required  in  the  PTEA 

•  Resource  availability  boundaries  for  analyses  conducted 
in  the  TAM 

•  Data  used  to  identify  potential  training  system  problems 
Overview  of  the  Functioning  of  the  TCM 

Figure  3  provides  a  graphic  overview  of  the  TCM.  The 
primary  inputs  to  the  TCM  are: 

•  Training  resource  availability  data 

•  Predecessor  system  training  data 

•  Training  technology  data  from  PM-TRADE  and  TTA 

•  Manpower  and  personnel  data  from  Product  3 

The  TCM  uses  this  input  to  generate  a  series  of  constraint 
data  bases.  The  data  is  used  in  the  TAM  to  generate  the  "most 
likely”  training  output  matrix  for  each  task  identified  by 
Product  X  of  the  MPT  Estimation  tools.  In  this  sense,  the  TCM  is 
a  preprocessor  of  data  for  the  TAM. 

The  TCM  js  exercised  to  develop  three  major  data  bases 
related  to: 

•  Training  resource  availability 

•  Predecessor  system  training  resource  consumption  and 
problems 
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Figure  3.  Training  constraints  module 


•  Likely  availability  of  new  training  technology  and 
methods 

The  three  major  elements  of  the  TCM  each  represent  a  data 
base  development  function.  In  each  case,  the  user  will  be  guided 
by  prompts  on  a  computer  to  identify  the  structure  of  the  data 
base  and  input  information  into  the  data  base.  Each  of  the  major 
elements  of  the  TCM  also  contains  user  help  functions  to  guide 
the  user  to  potential  sources  of  information  to  aid  in  completing 
the  data  bases.  The  three  major  elements  of  the  TCM  are 
described  in  more  detail  below. 


Ee  source  Availability  Data  Base 

The  resource  availability  data  base  is  the  principal 
component  of  the  TCM.  The  data  base  will  contain  estimates  of 
the  availability  of  critical  training  resources  for  the  emerging 
materiel  system  of  interest.  These  availability  limits  are  used 
to  set  the  boundaries  for  the  estimate  of  "most  likely"  training 
produced  by  the  TAM.  Table  1  summarizes  the  data  sources, 
structure,  user  inputs,  user  guidance,  and  output  of  the  resource 
availability  data  base.  Each  of  these  elements  of  the  data  base 
is  described  below. 

Data  Sources.  Preliminary  research  and  discussions  with 
SKEs  in  Army  training  development  and  system  acquisition 
processes  indicate  that  the  data  for  the  resource  availability 
data  base  must  be  obtained  from  different  sources  depending  on 
the  training  setting  involved.  The  proponent  TRADOC  combat 
development  center  or  school  for  the  emerging  materiel  system 
will  have  the  information  required  to  answer  resource 
availability  questions  concerning  resident  training.  The  \ 
training  elements  of  the  headquarters  of  the  major  U.S.  Army 
commands  of  the  "field  army"  such  as  FORSCOM,  USAREUR,  and 
WESTCOM  can  provide  much  of  the  data  concerning  training  resource 
limits  for  on-the-job  training  in  garrison.  Additional 
information  on  training  facilities,  particularly  facilities  which 
are  used  by  units  from  different  installations  such  as  the 
National  Training  Center  (NTC)  can  be  obtained  from  the  office  of 
the  Chief  of  Engineers  at  the  Department  of  Army.  Projections  on 
availability  of  new  technology  and  simulator  devices  may  be 
obtained  from  PM-TRADE  and  the  Training  Technology  Agency  (TTA) 
within  TRADOC. 

The  majority  of  the  training  resource  availability  data 
exists  in  written  documents  and  records  at  the  various  TRADOC 
schools,  TRADOC  HQ,  DA,  and  other  elements  of  the  Army.  Portions 
of  the  data  exist  in  electronic  media  as  part  of  the  Army 
Extension  Training  Information  System  (AETIS)  and  the  Army 
Instructional  Management  System  (AIMS) .  These  two  systems  both 
contain  training-related  data  bases  that  include  training 
resource  information.  This  data  is  related  primarily  to 
institutional  training,  however.  Part  of  the  development  process 
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Summary  of  the  training  resource  availability  data  base 
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of  the  TEM  will  be  the  further  specification  of  electronic  data 
bases,  agencies,  documents,  and  potential  POCs  from  which  the 
user  may  obtain  data  related  to  training  resource  availability. 

Structure .  Immediately  after  activating  the  resource 
availability  data  base  program,  the  user  will  be  queried  to 
provide  the  structure  for  the  data  base.  The  data  base  will  be 
constructed  as  a  three  dimensional  array  with  dimensions 
corresponding  to: 

•  training  setting 

•  time  dimension 

•  types  of  training  resources 

Figure  4  provides  an  illustration  of  the  data  base  structure 
with  sample  categories  listed. 

The  training  setting  dimension  has  three  default  categories 
which  are  associated  with  different  types  of  training  resources. 
The  default  categories  include: 

•  resident  training 

•  on-the-job  (garrison) 

•  tactical  field  training 

The  user  will  be  queried  to  accept  the  default  values, 
delete  irrelevant  settings,  or  further  subdivide  each  of  the 
settings  to  identify  more  specific  training  locations. 

The  time  period  dimension  of  the  resource  constraints  data 
base  will  have  a  default  value  of  l  year  increments  based  on  the 
government's  fiscal  year  calendar.  The  user  must  specify  the 
beginning  and  ending  fiscal  years  for  the  period  of  interest.  If 
desired,  the  user  may  use  calendar  years  rather  than  fiscal 
years. 

The  most  complex  dimension  to  be  established  in  the  data 
base  will  be  the  training  resource  type  dimension.  The  major 
types  of  resources  to  be  included  as  default  values  which  have 
been  identified  to  date  include: 

•  Classrooms 

•  Furnishings 

•  Instructors 

•  Materiel  System  Requirements 

•  Non-materiel-System  Equipment  Requirements 
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•  Range  requirements 

•  Maneuver  Areas 

•  Evaluators 

•  Simulation  devices 

•  Ammunition 

•  Opposing  Force  (OPFOR) 

The  program  will  allow  the  user  to: 

•  input  additional  types  of  training  resources  not  included 
in  the  default  categories 

•  subdivide  categories  such  as  instructors  and  classrooms 

•  to  delete  types  of  resources  not  relevant  to  the  materiel 
system  of  interest 

It  is  important  to  note,  however,  that  help  from  the  data 
base  program  regarding  suggested  sources  of  resource  constraints 
data  will  be  available  only  for  the  major  resource  types 
contained  as  default  values  in  the  program. 

Once  the  user  has  selected  the  parameters  to  be  used  for 
each  of  the  three  dimensions  of  the  constraints  data  base,  the 
program  will  generate  the  user's  data  base  structure.  The 
program  will  then  enter  the  next  phase  of  the  data  base 
development  process  and  begin  prompting  the  user  for  resource 
constraints  data  for  each  cell  of  the  data  base. 

User  Input.  The  initial  concept  for  the  TCM  was  based  on 
the  premise  that  an  initial  resource  availability  data  base  would 
be  developed  at  the  time  the  TEM  was  developed.  Experience 
gained  in  working  with  development  of  a  model  for  early-on 
training  analysis  for  the  Army's  Light  Helicopter  Family  (LUX) 
program  has  led  the  authors  to  conclude  that  training  resource 
constraints  data  must  be  input  by  the  user  specifically  for  the 
materiel  system  of  interest.  This  allows  the  data  base  to  be 
tailored  to  the  needs  of  the  user  and  avoids  the  need  for  a 
central  data  base  manager  who  must  constantly  update  the  TEM  data 
bases. 

The  user  will  be  prompted  to  complete  each  cell  of  the  three 
dimensional  data  base.  The  entry  in  each  cell  will  represent  the 
availability  for  a  particular  type  of  training  resource,  in  a 
specified  training  setting,  at  a  specific  point  in  time.  The 
initial  data  entry  prompt  will  be  for  the  input  of  the  extant 
limit  of  a  training  resource  category.  This  prompt  will  differ 
across  resource  categories.  For  example,  if  the  user  selects  the 
firing  range  class  as  a  relevant  resource  the  program  may  prompt 
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the  user  for  number,  size,  and  time  available  for  training  on  the 
ranges.  The  prompt  for  the  maneuver  area  is  likely  to  include 
input  values  for  area  available  and  time  available  while  the 
prompt  for  the  instructor  resource  will  probably  ask  only  for  the 
number  of  instructors  available. 

If  the  user  lacks  data  to  complete  a  cell  in  the  data  base 
he  may: 

•  move  to  another  cell  in  the  data  base  and  add  the  data 
for  the  cell  later 

•  query  the  program  for  suggested  data  sources 

•  input  a  "data  not  available"  entry 

•  identify  the  cell  as  not  applicable 

After  completing  the  input  for  a  prompt  for  the  extant 
resource  limit,  the  user  will  be  prompted  to  input  anticipated 
changes  in  resource  levels  during  the  time  period  of  interest. 

The  user  may  choose: 

•  a  steady  state  of  the  resource  limit  (no  change) 

•  input  specific  absolute  increases/decreases  by  year 

•  input  a  total  percentage  increase/decrease  across  the 
time  period  of  interest 

•  input  yearly  percentage  increases/decreases 

If  the  user  enters  a  single  percentage  change  of  resources 
value  for  the  entire  period  of  interest,  the  change  will  be 
distributed  evenly  across  the  entire  time  period.  It  should  be 
noted  that  a  user  may  enter  a  "zero"  availability  value  for  the 
initial  extant  resource  limit  prompt  and  then  increase  the 
availability  of  resources  above  zero  during  later  time  periods  of 
interest.  Regardless  of  the  format  of  user  entry,  each  entry 
stored  in  the  matrix  will  be  expressed  in  terms  of  the  absolute 
value  of  the  resource  availability. 

Guidance  to  users  on  data  sources.  It  is  anticipated  that 
TEM  users  may  not  initially  possess  all  of  the  information 
required  to  complete  the  training  constraints  data  base.  Thus, 
one  function  of  the  resource  constraints  data  base  element  of  the 
TCM  is  to  provide  guidance  to  the  user  on  how  to  obtain  such 
data.  In  its  final  form,  the  program  will  direct  the  user  to 
types  of  documents,  electronic  data  bases,  or  agency  points  of 
contact.  The  potential  sources  of  data  and  samples  of  the  type 
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of  training  resource  data  available  from  each  source  identified 
to  date  include: 

•  Proponent  Combat  Development  Centers 

course  hours 

-  instructor  projections 

-  materiel  system  availability 

•  TRADOC  Schools 

-  classrooms 

-  non-materiel  equipment 

-  furnishings 

-  instructors 

•  TRADOC  Headquarters 

-  instructor  projections 

-  facility  projections 

-  major  equipment  projections 

•  PM-TRADE 

new  simulation  technology 

•  FORSCOM,  USAREUR,  etc. 

combat  simulation  training  devices 
evaluators 

-  OPFOR 
ranges 

-  maneuver  areas 
ammunition 

•  Chief  of  Engineers  (DA) 

-  maneuver  areas 

-  major  training  facilities 

-  ranges 

•  AETIS  and  AIMS  data  bases 

course  hours 
equipment 

Output.  When  the  user  has  completed  running  the  program  for 
the  resource  availability  d3ta  base,  he  will  have  constructed  a 
detailed  data  base  indicating  maximum  limits  for  a  variety  of 
training  resources.  Furthermore,  the  data  base  will  provide 
information  regarding  expected  changes  in  these  resource  limits 
over  the  time  period  of  interest  to  the  user.  This  data  base 
will  be  particularly  important  when  used  in  conjunction  with  the 
training  resource  consumption  matrix  which  will  be  generated  as 
part  of  the  analytic  process  used  by  the  TAM  to  estimate  the 
"most  likely  training"  for  the  materiel  system. 
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Predecessor  Training  System  Data  Bases 


Given  the  evolutionary  nature  of  Army  training,  it  is 
essential  that  the  TEM  users  conduct  a  thorough  evaluation  of  the 
the  training  programs  for  predecessor  systems.  The  second  major 
element  of  the  TCM  is  composed  of  data  collection  guidance  and 
data  base  development  programs  to  construct  two  predecessor 
training  system  data  bases.  This  element  of  the  TCM  has  three 
major  components: 

•  A  user  training  program 

e  A  data  base  development  program  for  identifying 
predecessor  training  system  resource  requirements 

•  A  data  base  development  program  for  predecessor  training 
system  problems 

Each  of  these  components  is  discussed  in  more  detail  below. 

User  Data  Collection  Training.  The  first  component  of  the 
predecessor  system  data  base  element  of  the  TCM  will  be  a  user 
training  program.  The  program  will  provide: 

•  information  on  potential  data  sources 

•  questions  to  guide  data  collection 

•  sample  data  collection  forms 

The  purpose  of  the  on-line  training  program  is  to  aid  the 
user  in  obtaining  predecessor  training  system  data  in  the  most 
time  efficient  manner.  The  potential  data  sources  for 
development  of  the  two  predecessor  training  system  data  bases  are 
identified  later  in  this  section.  The  training  program  will 
provide  the  user  with  a  plan  of  attack  and  the  tools  needed  to 
collect  and  evaluate  predecessor  training  system  documentation. 
The  sample  questions  provided  in  the  training  program  will  help 
the  user: 

•  identify  the  most  likely  sources  of  data 

•  identify  the  categories  to  be  used  in  structuring  the 
predecessor  system  data  bases 

•  suggest  efficient  ways  of  extracting  and  summarizing  data 
from  existing  documentation 

The  training  program  will  provide  the  user  with  sample  data 
collection  or  data  summary  forms  which  may  be  used  to  summarize 
data  in  predecessor  training  system  documents.  The  program  will 
also  aid  the  user  in  constructing  customized  resource  requirement 
data  collection  forms  to  be  used  with  the  predecessor  systems  of 
interest.  The  content  of  the  program  will  be  based  on  lessons 
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learned  by  the  current  research  team  from  previous  experience  in 
development  of  predecessor  training  system  data  bases  used  for 
MPT  analyses. 

Predecessor  Resource  Requirements  Data  Base  Development 

The  predecessor  resource  requirements  data  base  element  of 
the  TCM  will  function  in  a  manner  similar  to  the  training 
resource  constraints  data  base  described  previously.  Table  2 
summarizes  the  data  sources,  structure,  user  inputs,  and  output 
for  the  predecessor  resource  requirements  data  base.  Each  major 
element  of  the  data  base  is  discussed  in  detail  below. 

Data  Sources.  Typically,  a  substantial  amount  of 
documentation  on  predecessor  training  system  resource 
requirements  exists  at  various  locations  throughout  the  Army. 

The  challenge  is  usually  that  of  collecting  and  digesting  the 
information.  Potential  sources  of  data  on  predecessor  training 
system  resource  requirements  which  have  been  used  by  the  current 
research  team  to  develop  predecessor  training  system  requirements 
data  bases  include: 

•  POIs  for  courses  for  relevant  MOS 

•  ICTPs  for  predecessor  systems 

•  Unit  Training  Plans 

•  ARTE PS 

•  Soldier  Manuals 

Additional  data  related  to  predecessor  training  system 
requirements  may  also  exist  on  a  variety  of  computerized  training 
management  and  information  systems.  The  Army  Extension  Training 
Information  System  (AETIS)  and  the  Army  Instructional  Management 
System  (AIMS)  are  potential  sources  of  such  data.  The  structure 
of  the  AIMS  is  still  under  development  and  may  include  a  variety 
of  data  bases  which  can  be  used  in  future  analyses  of  training 
systems.  It  should  be  noted,  however,  that  the  information 
contained  in  both  systems  is  related  primarily  to  training  in  the 
TRADOC  school  system. 

Structure.  Like  the  resource  availability  data  base,  the 
user  will  be  prompted  to  define  categories  on  the  two  dimensions 
of  the  predecessor  resource  requirements  data  base.  The  two 
dimensions  of  the  current  data  base  are  type  of  training  resource 
and  phase  or  element  of  the  predecessor  training  system. 

The  "type  of  training  resource"  dimension  in  the  current 
data  base  is  identical  to  the  training  resource  category 


E2-32 


E2-33 


dimension  the  user  defined  in  the  resource  availability  data 
base.  The  default  categories  for  the  dimension  are: 

•  Classrooms 

•  Furnishings 

•  Instructors 

•  Materiel  System  Requirements 

•  Non-materiel-System  Equipment  Requirements 

•  Range  requirements 

•  Maneuver  Areas 

•  Evaluators 

•  Simulation  devices 

•  Ammunition 

•  Opposing  Force  (OPFOR) 

The  user  may  delete,  add,  or  subdivide  resource  categories 
as  described  for  the  previous  data  base.  The  user  should  select 
the  same  resource  categories  on  this  dimension  of  the  predecessor 
resource  requirements  data  base  as  are  used  in  the  resource 
availability  data  base. 

The  second  dimension  representing  the  phase  or  element  of 
the  predecessor  training  system  is  unique  to  the  current  data 
base.  This  dimension  requires  the  user  to  define  new  categories. 
The  user  will  be  prompted  to  define  relevant  phases  or  segments 
of  three  types  of  training  for  the  predecessor  system  including: 

•  Institutional  training  courses 

•  Unit  training  in  garrison 

•  Capstone  tactical  training  events  such  as  NTC 

The  user  will  have  received  information  concerning  how  to 
define  such  phases  of  training  in  the  user  training  program. 

While  the  institutional  courses  and  capstone  tactical  training 
events  are  self-explanatory,  the  unit  training  categories  are 
somewhat  more  difficult  to  define.  In  development  of  predecessor 
training  system  requirement  data  base  for  research  on  the  LUX, 
the  phases  of  predecessor  unit  training  included  in  the  data  base 
were: 


•  new  equipment  training  (NET) 
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•  individual/crew  training 

•  company/unit  training 

•  gunnery  training 

•  battalion  training 

•  ARTEP 

'  User  input.  Once  the  structure  for  the  data  base  has  been 
defined,  the  user  will  be  prompted  to  complete  each  cell  of  the 
matrix.  The  user  entries  will  be  absolute  values  of  the 
resources  required  to  implement  each  phase  of  the  predecessor 
training  system.  The  data  will  represent  resources  consumed 
during  a  single  cycle  of  a  specific  phase  of  the  predecessor 
system  training  program  rather  than  total  resources  consumed. 
There  is  no  projection  of  resource  requirements  over  time  for  the 
predecessor  training  requirements  data  base.  The  resource 
requirements  are  calculated  for  a  single  training  cycle. 

Output.  Figure  5  provides  an  illustration  of  the  output 
structure  of  a  sample  predecessor  resource  requirements  data 
base.  The  predecessor  resource  requirements  data  base  will  be 
used  in  conjunction  with  output  from  the  TAM  to  construct  a 
resource  requirements  matrix  for  the  overall  training  program  for 
the  material  system  being  evaluated  by  the  TEM  user.  Greater 
detail  on  the  use  of  this  data  base  will  be  described  in  the 
description  of  the  TAM. 

Predecessor  Training  System  Problems 

The  data  base  for  the  predecessor  training  system  problems 
represents  a  qualitative  data  base  rather  than  a  quantitative 
data  base.  The  construction  of  the  data  base  will  be  interactive 
as  was  the  case  with  the  previous  data  bases  discussed.  Table  3 
summarizes  the  data  sources,  structure,  user  inputs,  and  output 
for  the  predecessor  training  system  problem  data  base.  Each 
element  in  the  table  is  described  below. 

Data  Sources.  There  are  fever  written  sources  of 
information  for  predecessor  training  system  problems  than  for  the 
other  data  bases  included  in  the  TCM.  Potential  Sources  for  data 
on  predecessor  training  system  problems  identified  to  data 
include: 


•  Post-fielding  CTEAs 

•  MAAs  conducted  after  fielding  of  the  system 

•  Interviews  with  SMEs 
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Figure  5.  Sample  predecessor  resource  requirements  data  base  structure 
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The  identification  of  additional  sources  of  potential  data 
for  this  data  base  represents  a  task  which  must  be  completed  in 
later  phases  of  development  of  the  TEM. 

Structure.  The  structure  of  the  current  data  base  is  not 
specified  apriori  by  the  user  as  was  the  case  with  the  previous 
data  bases.  Rather,  the  data  base  will  be  structured  as  a  series 
of  text  files  to  be  searched,  sorted,  and  printed  by  a  data  base 
program.  The  size  of  the  data  base  will  depend  on  the  number  of 
questions  answered  by  the  user  and  the  length  of  the  answers 
provided.  The  underlying  file  structure  of  the  data  base 
includes  four  data  fields  corresponding  to  the  user  prompt 
questions  for  each  potential  problem  area.  The  data  fields 
include: 


•  relevance  of  the  problem  area 

•  indication  of  presence  of  problem 

•  description  of  the  problem 

•  source  of  data  describing  the  problem 

The  data  base  will  have  a  minimum  default:  number  of  text 
files  corresponding  to  the  number  of  problem  area  questions  used 
as  user  prompts.  The  data  base  program  will  be  structured  to 
allow  the  user  to  enter  additional  problem  areas  with  their 
associated  data. 

User  Input.  The  user  will  be  prompted  with  a  set  of 
questions  regarding  potential  problem  areas  which  may  have  been 
identified  for  the  predecessor  system.  For  each  potential 
problem  area,  the  user  will  input: 

•  Whether  the  problem  area  is  relevant  to  the  predecessor 
system 

•  Whether  the  presence  of  a  problem  in  that  area  was 
identified  (if  the  problem  area  is  relevant) 

•  A  short  description  of  the  problem  if  one  was  identified 
to  include  as  much  quantitative  data  as  possible 

•  Notation  of  the  data  source 

The  utility  of  the  data  base  will,  of  course,  depend  heavily 
on  the  questions  asked  of  the  user.  Given  the  emphasis  of  the 
current  effort  on  providing  interface  designers  with  potentially 
useful  data,  many  of  the  questions  will  be  constructed  to  query 
the  user  concerning  "training  problems"  which  might  be  related  to 
design  problems  with  the  predecessor  system.  Potential  areas  for 
questions  include: 

•  Negative  transfer  of  training  effects 
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•  Training  errors  or  accidents  due  to  design  problems 

9  Crew  training  problems  due  to  limited  communication  or 
difficult  positions 

#  Unit  training  problems  in  movement  formation  due  to 
limited  visibility,  etc 

Output .  The  predecessor  training  system  data  base 
development  program  will  include  an  option  to  print  a  listing  of 
the  user  prompt  questions.  This  listing  of  questions  may  be  used 
to  develop  data  collection  instruments  in  the  form  of  surveys  or 
interview  protocols  to  be  used  with  SMEs  who  have  conducted 
extensive  training  with  the  predecessor  system. 

The  predecessor  training  system  problem  data  base  will  be 
used  with  the  FIM  to  generate  structured  qualitative  reports  for 
interface  designers.  The  data  base  will  have  key  word  and  key 
phrase  search  and  sort  capabilities  and  provide  the  data  source 
citations  required  to  form  an  audit  trail  to  seek  more 
information  regarding  the  relevance  of  the  predecessor  system 
problem  to  the  materiel  system  of  interest  to  the  user  of  the  TEM 
output . 


New  Training  Technology  Constraints 

A  major  factor  which  may  cause  a  change  in  a  new  system's 
training  program  from  that  of  predecessor  training  systems  is  the 
availability  of  new  training  technology.  It  must  be  recognized 
that  a  number  of  factors  will  impact  on  the  availability  of  such 
technology  including  biases  of  the  proponent  schools  in  which 
training  will  occur. 

The  purpose  of  the  third  major  element  of  the  TCM  is  to 
identify  factors  which  might  limit  the  use  of  new  training 
methods  or  technologies  for  the  emerging  materiel  system.  For 
the  purposes  of  estimating  the  "most  likely  training"  program,  a 
new  training  technology  or  method  is  defined  as  a  training  method 
or  technology  which  has  never  been  applied  in  training  for 
predecessor  systems.  While  the  technology  may  have  existed  for 
some  time,  it  will  be  considered  as  new  for  the  training  system 
under  analysis. 

Table  4  summarizes  the  data  sources,  structure,  user  input, 
and  output  of  the  technology  constraints  data  base. 

The  development  of  the  technology  constraints  data  base  will  be 
accomplished  in  three  phases  as  listed: 

9  Identification  of  potential  new  training  methods  and 
technologies 

9  User  response  to  technology  availability  questions 
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Summary  of  the  new  technology  constraints  data  base 
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•  Identification  of  new  training  methods  and  technologies 
to  be  considered  as  "available"  for  the  new  materiel 
system  training  program 

The  three  phases  of  the  data  base  program  described  above 
correspond  roughly  to  development  of  the  data  base  structure, 
user  input,  and  the  data  base  output  functions.  The  potential 
sources  of  data  to  be  used  for  this  data  base  and  the  other  major 
elements  of  the  data  base  are  described  below. 

•  Potential  Data  Sources.  The  identification  of  new  training 
methods  and  technologies,  as  defined  in  the  context  of  this  data 
base,  will  require  two  types  of  information.  The  first  type  of 
data  sources  are  those  providing  information  about  the 
predecessor  training  system.  These  data  sources  are  identical  to 
the  sources  used  for  the  predecessor  training  resource 
requirements  data  based  described  previously. 

The  second  type  of  data  sources  required  are  those  related 
to  potential  training  methods  and  technologies  which  have  not 
been  used  with  predecessor  systems.  The  TAM  provides  a  variety 
of  potential  training  methods  to  the  user  and  will  be  a  principal 
source  of  data  for  potential  instructional  methods.  Information 
regarding  the  latest  simulators  and  other  high  technology 
training  applications  can  be  obtained  by  the  user  from  PM-TRADE 
or  from  the  TTA  in  TRADOC. 

Structure.  As  noted  above,  the  first  step  in  the 
development  of  the  current  data  base  will  involve  identification 
of  potential  training  technologies  which  have  not  been  used  in 
predecessor  training  systems.  By  comparing  information  from  the 
data  sources  listed  above,  the  user  can  compile  a  list  of 
potential  new  training  methods  and  technologies.  This  list  can 
be  compiled  manually  or  a  routine  in  the  data  base  program  can  be 
used  to  prompt  the  user  for  a  variety  of  potential  training 
technologies  and  methods.  If  the  prompts  are  used,  the  user  will 
simply  indicate  if  the  training  method  had  been  used  in 
predecessor  systems.  The  user  will  then  add  any  additional 
training  methods  identified  during  his  data  collection  effort. 
Alternatively,  the  user  may  enter  his  own  list  of  training 
methods  and  technologies,  bypassing  the  prompts  entirely.  In 
either  case,  th^.  list  of  new  potential  training  methods  and 
technology  will  establish  the  structure  for  the  remainder  of  the 
data  base.  The  data  base  program  will  prompt  the  user  to  answer 
a  series  of  questions  for  each  method  or  technology  included  in 
the  list  entered  in  this  phase  of  the  program. 

User  Input.  Once  the  user  has  identified  the  potential 
training  methods  and  technologies  not  previously  used  in 
predecessor  system  training,  the  data  base  program  will  prompt 
the  user  to  answer  a  series  of  questions  concerning  each  training 
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method  or  technology.  The  questions  to  be  answered  by  the  user 
for  this  data  base  are  listed  below. 

1.  Does  the  proponent  school  or  units  receiving  the  new 
system  currently  use  the  training  method  or  technology  with  other 
systems?  (YES  OR  HO) 

2.  Do  other  schools  or  units  in  the  Army  currently  use  the 
training  method  or  technology?  (YES  OR  NO) 

3.  Are  the  proponent  school  or  units  receiving  the  new 
system  currently  authorized  and  funded  to  purchase  or  develop  the 
new  training  methods  or  technology?  (YES  OR  NO) 

4.  Have  the  schools  or  units  requested  the  funds  or  other 
resources  required  to  purchase  or  develop  the  technology  or 
training  method?  (YES  or  NO) 

5.  Is  the  training  method  or  technology  currently  available 
in  the  commercial  market?  (YES  OR  NO) 

6.  If  the  new  technology  or  method  is  not  available  in  the 
Army  or  commercial  market,  what  is  the  current  stage  of 
development  of  the  technology: 

a.  In  production 

b.  Prototype  testing 

c.  Prototype  development 

d.  Experimental  testing  of  method 

e.  Basic  research  and  development 

f.  Concept  development  phase. 

7.  If  the  training  technology  or  method  were  available  at 
no  cost  to  the  proponent  school  or  units  receiving  the  new 
materiel  system  what  is  your  estimate  of  the  probability  that 
they  would  use  the  technology  or  method  in  their  training 
program?  (enter  a  probability  value  from  .00  to  1.00) 

Output.  The  data  base  program  will  "score"  each  question 
and  calculate  a  total  score  scaled  to  reflect  the  "probability  of 
use"  for  each  new  training  technology  or  method  in  the  data  base. 
The  most  appropriate  "cut-off"  score  for  a  training  method  or 
technology  will  be  determined  during  the  development  phase  of  the 
TEM.  Based  on  this  cut-off  score,  the  technology  constraints 
program  will  output  a  list  of  "likely  new  training  technologies 
or  methods"  which  should  be  considered  as  feasible  for  use  in  the 
training  program  for  the  materiel  system  of  interest.  New 
training  technologies  and  methods  which  are  not  included  in  the 
list  will  be  excluded  from  consideration  in  development  of  the 
"most  likely  training  program"  constructed  using  the  TAM. 
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Additional  Training  constraint  para  Bases 

In  addition  to  the  three  major  elements  described  above, 
the  TCM  will  guide  the  user  through  an  examination  of  one,  and 
possibly  two,  additional  sets  of  constraints  on  the  "most  likely" 
training  program  for  the  materiel  system  of  interest.  These  two 
additional  sets  of  constraints  are  briefly  discussed  below. 


Manpower- and  Personnel  Constraints 

Product  3  of  the  MPT-Estimation  Tools  will  provide  a  lower 
bound  estimate  of  the  target  audience  for  the  materiel  system  of 
interest.  The  TCM  will  include  a  data  base  of  this  output.  This 
information  will  be  used  by  SMEs  during  analyses  conducted  using 
the  TAM.  An  additional  source  of  manpower  and  personnel 
constraints  data  which  is  relevant  for  the  purposes  of  estimating 
the  most  likely  training  program  will  be  an  estimate  of  the 
number  of  trainees  who  will  be  in  the  active  versus  reserve 
components  of  the  Army.  The  TEM  user  will  be  guided  to  examine 
the  materiel  system  distribution  plan  to  derive  an  estimate  of 
the  number  of  reserve  personnel  who  must  be  trained  for  the 
system. 


serve  Component  Training  Constraints 

Wiile  the  SOW  does  not  address  the  issue  of  active  vs. 
reserve  personnel,  the  increasing  role  which  the  reserve 
components  of  the  Army  are  expected  to  play  in  the  event  of  war, 
suggests  that  this  may  be  a  factor  which  must  be  cr.sidered. 

This  reserve  component  issue  is  particularly  important  in 
providing  the  interface  designer  and  training  system  developer 
with  an  accurate  estimate  of  the  most  likely  training  to  be 
received  by  personnel  using  the  system.  Reserve  personnel  often 
spend  less  time  in  resident  training,  have  "institutional 
training"  distributed  over  longer  time  periods,  train  less 
frequently,  and  experience  other  differences  in  training  compared 
to  active  component  personnel  in  the  Army.  This  difference  in 
training  experience  is  particularly  true  for  hands-on  training 
and  for  maintenance  training  of  major  systems. 

The  data  bases  of  the  TCM  may  be  developed  for  both  active 
and  reserve  components  to  provide  the  TEM  user  with  the  data 
required  for  comparison  of  the  most  likely  training  systems  which 
will  evolve  for  the  personnel  in  the  two  components.  This 
information  may  be  critical  if  reserve  personnel  are  expected  to 
man  the  system  during  periods  of  war  and  if  the  most  likely 
training  systems  for  the  active  and  reserve  component  appear  to 
differ  substantially. 
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Summary_of  the  TCM 


The  purpose  of  the  TCM  is  to  develop  a  set  of  training 
constraints  data  bases  which  will  be  used  as  input  for  the 
Training  Analysis  Module  (TAM)  and  to  generate  reports  relevant 
to  the  PTEA.  The  TCM  consists  of  three  major  elements  including: 

•  Development  of  a  training  resource  constraints  data  base 

•  Development  of  predecessor  training  system  resource 
requirements  and  training  system  problem  data  bases 

•  Identification  of  constraints  on  the  application  of  new 
training  technology  and  methods  for  the  emerging  materiel 
system  training  program 

The  TCM  uses  computer  programs  to  guide  the  user  through  the 
development  of  the  data  base  structure  and  to  input  data  for  each 
of  the  functions  identified  above.  The  programs  for  each  of  the 
three  functions  provide  some  form  of  help  to  the  user  in 
identifying  potential  sources  of  information  needed  to 
development  the  data  base. 

The  completion  of  the  three  major  functions  of  the  TCM 
results  in  the  development  of  four  data  bases.  The  data  bases 
include: 


•  Training  resource  constraints 

e  Predecessor  system  training  resource  requirements 

•  Predecessor  training  system  problems 
e  Training  technology  constraints 


The  Training  Analysis  Module 


Objective 

As  noted  in  previous  sections,  the  Training  Estimation 
Methodology  is  being  developed  to  support  the  conduct  of  PTEAs. 

In  this  capacity,  the  TEM  must  address:  (1)  training 
constraints,  (2)  potential  training  problem  areas,  and  (3)  the 
development  of  the  training  concept  for  an  emerging  materiel 
system.  TRADOC  Regulation  350-4,  The  TRADOC  Training 
Effectiveness  Analysis  f TEA)  System,  states  that  the  training 
concept  is  to  delineate  the  following  aspects  of  an  emerging 
system's  training  program:  (1)  what  is  to  be  trained;  (2)  who  is 
to  be  trained;  and  (3)  the  "how,  when,  and  where"  of  training. 

The  Constraints  module  described  in  the  previous  section 
addresses  training  constraints  and  problem  areas.  Training 
concept  development  is  the  objective  of  the  Analysis  module,  or 
TAM. 
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The  training  concept  for  an  emerging  system  outlines  the 
broad  architecture  of  the  associated  training  program.  Later, 
more  detailed  training  analysis  and  design  activities  carried  out 
as  part  of  the  CTEA  process  will  use  the  training  concept  as  a 
point  of  departure  for  the  definition  of  MOS-specific  training 
requirements.  In  our  view,  a  training  system  is  a  process 
structured  to  impart  necessary  job  skills  to  a  specified  target 
population.  In  developing  the  TAM,  we  are  attempting  to 
ioimulatc'  a  iaethcdclcgy  that  vi  11  assist  Army  training  developer- 
in:  (1)  early-on  thinking  about  training  as  a  system  in  which  a 

complex  network  of  skills  is  acquired  in  a  progressive  fashion 
using  cost-effective  means,  and  (2)  formally  defining  the  scope 
and  general  nature  of  the  most  suitable  system  for  meeting  this 
end. 


When  the  output  of  the  TAM  is  analyzed  within  the  context  of 
the  constraints  identified  in  the  TCM,  the  output  represents  the 
"most  likely"  training  program  to  emerge  for  the  materiel  system 
of  interest.  This  information  will  be  of  greatest  interest  to 
the  materiel  designers  seeking  an  estimate  of  the  training 
program  which  will  accompany  the  system  they  design. 

The  TAM  Concept 

A  graphic  overview  of  the  TAM  is  presented  as  Figure  6. 

From  Figure  6,  input  to  the  TAM  consists  of: 

•  The  output  from  Product  I: 

Behavioral  elements  (i.e.,  tasks) 

Conditions  of  performance 

Performance  standards 

•  System  Concept (s):  Ideas  concerning  what  the  proponent 
wants  to  acquire 

•  Information  regarding  predecessor  or  referent  systems 

•  Output  from  the  Constraints  module: 

The  constraints  boundary 

Training  problems  on  predecessor  or  referent  systems 

•  Output  from  Product  3:  A  Target  Audience  Description 

The  collective  output  from  Product  1  is  used  to  structure  a 
description  of  the  composite  performance  environment  of  the 
target  system:  doctrine,  tactics,  performance  demands  (tasks, 
conditions,  and  standards),  maintenance  and  support  concepts,  and 
the  like.  All  of  this  information  serves  as  the  data  base  for 
the  training-related  analyses  to  follow. 

The  TAM  is  exercised  in  five  elements,  listed  as  follows: 

•  Task  Identification 
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Figure  6.  The  training  analysis  module- 
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•  Training  Prescription 

•  Training  Priority,  Time,  and  Option 

•  Training  Program  Synthesis 

•  Training  Resource  and  Constraints  Assessment 

The  first  four  TAM  elements  are  similar  to  the  activities 
nerfonmed  as  Dart  of  a  conventional,  detailed  training 
requirements  analysis,  but  are  intended  to  function  at  a  generic 
as  opposed  to  a  specific  level.  Early-on  during  system 
development,  specific  information  regarding  materiel  items  and 
employment  concepts  is  often  not  available.  Hence,  to  be  useful 
to  training  developers  during  concept  exploration,  the  TAM  must 
be  capable  of  processing  generic  information. 

The  first  four  elements  comprising  the  TAM  are  adapted  from 
similar  procedures  developed  for  the  Training  Developers  Decision 
Aid  (TDDA)  Versions  1  and  2  (Pieper,  Guard,  Michael,  &  Kordek, 
1978;  Pieper,  Elliott,  &  Hawley,  1979;  Hawley,  1979;  and 
Frederickson,  Hawley,  &  Whitmore,  1983).  Both  previous  versions 
of  the  TDDA  were  developed  to  assist  Army  training  developers  in 
structuring  training  for  fielded  weapons  systems.  In  the  present 
application,  the  TDDA  methods  have  been  generalized  for  use  with 
concept-level  (i.e.,  pre-prototype  stage)  systems. 

The  fifth  element  of  the  TAM,  the  resource  and  constraints 
assessment  represents  a  deviation  from  earlier  versions  of  the 
TDDA.  While  the  first  four  elements  of  the  TAM  produce 
information  regarding  an  "optimal”  training  program,  the  fifth 
element  will  utilize  data  from  the  TCM  to  modify  the  "optimal" 
training  program  to  fit  projected  resource  and  technology 
constraints.  The  results  obtained  from  exercising  the  fifth 
element  is  the  TEM's  prediction  of  the  "most  likely  training" 
program  for  the  materiel  system  being  evaluated.  This  prediction 
reflects  the  impact  of  the  predecessor  training  system,  unique 
training  requirements  identified  for  the  new  system,  and 
constraint  factors  which  impact  on  the  training  system.  Each 
element  comprising  the  Analysis  module  is  now  described  in  turn. 

Task  jcfttiPD 

As  noted  previously,  one  of  the  inputs  to  the  TAM  (from 
Product  1)  consists  of  a  set  of  target  behavioral  elements  (i.e., 
tasks)  along  with  performance  standards  and  a  listing  of  the 
conditions  under  which  a  job  will  be  performed.  The  term 
"behavioral  elements"  is  used  here  instead  of  "tasks"  because  we 
are  not  certain  at  this  time  of  the  precise  nature  of  the  output 
from  Product  1.  Regardless  of  the  form  of  the  output  from 
Product  1,  it  is  likely  that  some  amount  of  "polishing"  will  have 
to  be  performed  on  the  resulting  output  prior  to  using  it  in  the 
Analysis  module.  The  objective  of  the  Task  Identification 
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element  is  to  serve  an  interface  between  Product  1  and  the  TAM; 
it  will  be  tailored  to  serve  in  this  role  whatever  the  eventual 
form  of  the  output  from  Product  1. 

As  the  first  step  in  exercising  the  TAM,  the  behavioral 
elements  received  from  Product  1  are  recast  as  task  statements 
suitable  for  use  with  the  module.  This  consists,  first,  of 
mapping  the  behavioral  elements  from  Product  1  onto  a  set  of  Task 
Action  Verbs  that  are  conformable  with  other  TAM  elements.  Table 
5  presents  a  set  of  representative  Task  Action  Verbs  used  in  the 
TAM.  A  complete  listing  of  Task  Action  Verbs  is  presented  in 
Appendix  A.  It  should  oe  noted  thaL  mapping  behavioral  elements 
onto  Task  Action  Verbs  may  require  the  user  to  parse  the  output 
from  Product  1  into  smaller  behavioral  units.  That  is,  function¬ 
like  statements  may  have  to  be  broken  down  into  sub-functions; 
and  sub-function-1 iKe  statements  may  have  to  be  broken  down  into 
behavioral  elements  resembling  tasks. 

Table  5. 

Representative  TAM  task  action  verbs 


Verb 

Abide  by 
Accept 

Awww.»..t*od3te 

Acquire 

Activate 

Adapt 

Adjust 

Adjust  to 

Advise 

Aim 

Align 

Allocate 

Analyze 

Answer 

Anticipate 


Verb 

Apply 

Arrange 

Assemble  (Dis) 

Assign 

Associate 

Attend 

Calculate 

Catalogue 

Categorize 

Characterize 

Check 

Choose 

Cite 

Classify 
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Clean 


Once  appropriate  Task  Action  Verbs  have  been  selected,  the 
next  step  in  creating  task  statements  involves  determining  system 
and  support  (i.e.,  stand-alone  electronic  test  equipment,  special 
tools,  etc.)  equipment  items.  System  and  support  equipment  items 
are  determined  on  the  basis  of  predecessor  or  referent  system 
configurations  (i.e.,  from  a  BCS) .  The  completed  task  statement 
then  consists  of  a  Task  Action  Verb,  an  item  of  system  equipment 
that  is  acted  on,  any  support  equipment  used,  the  performance 
standard,  and  conditions  of  performance.  Task  statements  are 
also  categorized  according  to  whether  they  pertain  to  operations, 
maintenance,  or  support.  All  of  the  steps  comprising  the  Task 
Identification  element  of  the  TAM  are  shown  graphically  in  Figure 
7. 

Tra i n i ng  _ Prescription 

The  second  element  comprising  the  TAM  involves  the 
specification  of  a  training  prescription  for  each  task  statement. 
Task  training  prescriptions  are  generated  through  a  logical  and 
analytical  process  that  results  in  t^e  specification  of  five 
items  of  information: 

•  Learning  Algorithm 

•  Stimulus  Medium 

•  Response  Acceptance  Mechanism 

•  Method  of  Instruction 

•  Learning  Setting 

Each  of  the  five  aspects  of  the  training  prescription  is 
described  in  the  subsections  to  follows. 

Learning  algorithm.  The  Task  Action  Verb  is  used  to 
uniquely  specify  one  of  12  learning  algorithms.  A  learning 
algorithm  is  a  model  instructional  approach  which  is  appropriate 
for  tasks  of  a  given  type.  As  an  example  of  a  learning 
algorithm,  Figure  8  presents  the  algorithm  for  tasks  involving 
•'Identifying  Symbols.”  The  learning  algorithms  proposed  for  the 
TAM  are  adapted  from  those  used  in  Version  1  of  the  TDDA,  which 
are  derived  from  algorithms  presented  in  TAEG  (Training  Analysis 
and  Evaluation  Group)  Reports  16  and  23  (Braby,  Henry,  Parrish,  & 
Swope,  1975;  Aagard  &  Braby,  1976).  These  learning  algorithms 
are  listed  and  defined  as  follows: 

1.  Identifying  Symbols.  Recognizing  and  interpreting 
graphic  characters  such  as  those  used  in  engineering  drawing, 
maps,  and  status  boards. 

2.  Verbal  Chaining.  Recalling  and  applying  previously 
learned  facts  and  principles  in  a  job  setting. 
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Figure  7.  Task  identification, 
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Figure  8 


Learning  algorithm  for  identifying  symbols, 
(adapted  from  Aagard  &  Braby,  1976) 
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Present  first/next  symbol 
from  set  in  contiguous 
association  with  referent 


E2-52 


•  PRESENT  RANDOMLY 
SELECTED  FIRST/NEXT 
SYMBOL  FROM  SET 

•  TEST  STUDENTS  TO  SEE  IF 
HE  CAN  IDENTIFY  SYMBOL 


STUDENT  OVERTLY  IDENTIFIES  SYflr" 


CRITERION; 

“X"  CONSECUTIVE  CQPP.Z:' 
IDENTIFICATIONS  FDR  EAI 
SYMBOL  (INCLUDING  OVER- 
LEARNING  REQUIREMENT) 
WITHOUT  REQUIRING  A 
MEMORY  AID.  DROP 
SYMBOL  FORM  SET  WHEN 
CRITERION  IS  ACHIEVED. 


Figure  8.  (continued) 
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Figure  8 


(continued) 
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3.  Rule  Using.  The  acquisition  and  application  of 
established  practices  or  fixed  principles  (rules)  that  serve  as 
guides  in  selecting  courses  of  action. 

4.  Classifying.  Assigning  names  to  detected  symbols  based 
on  identifiable  characteristics. 

5.  Decision  Making.  The  application  of  a  specific  decision 
model  thought  to  be  useful  in  diagnosing  equipment  malfunctions, 
choosing  tactics,  or  in  planning  where  several  alternatives  must 
be  considered. 

6.  Problem  Solving.  Evaluating  and  integrating  information 
from  a  number  of  sources  to  effect  a  solution  to  a  problem. 

7.  Gross  Motor  Skills.  Repetitive  motor  acts  involving 
chaining  of  movements  and  a  low  level  of  attention  by  a  skillful 
performer. 

8.  Motor  Chaining.  Carrying  out  routinized  activities, 
executed  as  standard  operating  procedures,  in  some  predetermined 
sequence . 

9.  Steering,  Guiding,  and  Continuous  Movement.  Continuous 
physical  responses  to  a  constantly  moving  visual  reference. 

10.  Communicating.  A  conversation  between  people  in  which 
standardized  message  formats  are  employed. 

11.  Monitoring.  Becoming  alert  to  the  presence  of  a  signal 
that  could  be  of  special  interest  in  the  performance  of  a  job  or 
mission. 


12.  Attitudes.  Learning,  recognizing,  and  recalling 
information  needed  to  function  in  an  operational  setting. 

Appendix  A  lists  each  Task  Action  Verb  along  with  its 
associated  learning  algorithm. 

Stimulus  Medium.  The  stimulus  medium  prescribes  the 
modality  or  mechanism  through  which  task  stimuli  are  to  be 
presented  to  trainees.  In  characterizing  the  stimulus  medium, 

TEM  users  first  identify  the  general  class  of  stimuli  that  will 
be  encountered  in  the  job  environment.  This  is  done  to  insure  as 
much  job-related  fidelity  as  possible  in  the  stimulus 
presentation  during  training.  The  general  classes  of  stimuli 
proposed  for  the  TAM  are  the  following: 

•  Verbal.  Letters  or  numbers  are  used  by  the  job  holder  in 
performing  the  task 

•  Audio.  Sounds  are  important  in  doing  a  task 
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•  Audio/Visual.  Sounds  and  visual  observations  are 
important  in  performing  a  task 

•  Visual.  Visual  observations  are  required  to  do  a  task 

•  Tactile.  The  job  holder  is  required  to  use  his  Bense  of 
touch  in  the  performance  of  a  task 

In  the  event  that  more  than  one  stimulus  class  is 
applicable,  users  should  select  all  relevant  classes  and  then 
process  the  logical  union  of  all  ensuing  results. 

The  user  next  describes  selected  characteristics  of  the  task 
stimulus  in  sufficient  detail  to  discriminate  between 
presentation  media  appropriate  for  various  stimulus  classes. 
Stimulus  characterization  items  are  listed  as  follows: 

•  Are  stimuli  in  color? 

•  Are  stimuli  equipment  indicators? 

•  Are  audio  stimuli  voice  only? 

•  Are  stimuli  visually  distinct  —  not  obscured  or 
overshadowed  by  peripheral  stimuli? 

•  Are  movements  continuous  as  opposed  to  discrete  (i.e., 
"on"  or  "off")? 

•  Are  verbal  stimuli  audio? 

•  Are  stimuli  frequently  changed  or  updated? 

•  Are  stimuli  three-dimensional  (i.e.,  solid  objects)? 

•  Are  many  ambient  sounds  present? 

•  Are  ambient  sounds  random  as  opposed  to  cyclic  or 
periodic? 

•  Are  audio  stimuli  equipment  generated? 

•  Do  the  stimuli  move? 

•  Will  performing  the  task  wrong  result  in  damage  to  system 
equipment? 

•  Is  the  system  equipment  expected  to  be  operational  at 
least  75%  of  the  time? 

•  Is  the  task  a  maintenance  or  support  task  (as  opposed  to 
an  operator  task)? 
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Responses  to  the  stimulus  characterization  items  are  used  to 
select  the  most  appropriate  media  to  present  stimulus 
characteristics  to  trainees.  The  logic  of  the  TAM  media 
selection  process  is  illustrated  in  Appendix  B,  Figures  B-l 
through  B-5.  The  following  stimulus  presentation  media  are 
currently  included  in  the  TAM: 

•  Audio  tape 

•  Printed  material 

•  Microform 

•  Mockup 

•  Movie 

•  Actual  Equipment 

•  Silent  film  strip 

•  Silent  slides 

•  Simulator 

•  Sound  film  strip 

•  Sound  slides 

•  Television 

Additional  stimuli  presentation  media  will  be  added  to  the 
TAM  during  the  development  phase  of  the  current  effort.  The 
media  to  be  adder  included  presentation  through  micro-computers 
(CRT)  and  computer  interactive  video  disc  (CIVD) . 

Response  acceptance  mechanism.  The  training  prescription 
process  continues  with  the  identification  of  a  response 
acceptance  mechanism — the  manner  in  which  a  trainee's  responses 
to  the  learning  situation  are  accepted  and  evaluated  and  feedback 
provided.  As  with  the  stimulus  medium,  the  user  first  identifies 
the  general  class  of  response  involved,  and  then  proceeds  to 
answer  a  series  of  questions  that  further  characterize  the  nature 
of  the  response.  The  general  classes  of  responses  employed  in 
the  TAM  are  given  as  follows: 

•  Equipment  Manipulation.  The  job  holder  must  physically 
manipulate  equipment  while  performing  a  task. 

•  Voice.  The  job  holder  is  required  to  use  his  or  her 
voice  while  performing  a  task. 

•  Written.  The  job  holder  is  required  to  record  or  write 
while  performing  a  task. 


E2-57 


•  Body  Movement.  The  job  holder  uses  specified  body 
movements  in  the  performance  of  a  task. 

The  items  used  to  further  characterize  trainee  responses  are 
listed  as  follows: 

•  Is  the  task  a  maintenance  or  support  task  (i.e.,  one  that 
requires  access  to  the  interior  of  the  system) ? 

•  Are  equipment  manipulations  discrete  as  opposed  to 
continuous? 

•  Are  equipment  control  displays  nonlinear? 

•  Will  performing  the  task  wrong  result  in  damage  to  system 
equipment? 

•  Is  the  system  equipment  operational  at  least  75%  of  the 
time? 

•  Does  the  response  consist  mainly  of  knowing  control  or 
display  names  and  locations? 

•  Does  the  student  respond  by  giving  instructions  or  orders 
to  a  group? 

•  Does  the  response  require  interactions  with  others? 

•  Does  the  job  performer  evaluate  his  or  her  own  response? 

•  Is  the  response  a  mix  of  selected  and  constructed? 

•  Is  the  response  selected,  or  mostly  selected? 

•  Is  the  response  written  on  a  form? 

•  Does  the  job  performer  respond  to  voice  instructions? 

•  Does  the  response  consist  of  a  coordinated  group 
performance? 

Response  acceptance  mechanisms  proposed  for  the  TAM  are 
listed  below.  The  logic  of  selecting  response  acceptance 
mechanisms  on  the  basis  of  user  input  is  illustrated  in  Appendix 
B,  Figures  B-6  through  B-10. 

•  Audio  tape  recorder 

•  Printed  material 

•  Group  instructor 

•  Mockup 
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•  Actual  equipment 

•  Simulator 

•  Teaching  machine 

•  Tutor 

•  Video  recorder 

The  teaching  machine  response  acceptance  mechanism  will  be 
modified  during  the  development  phase  in  the  current  effort  to 
represent  a  computer  terminal  or  micro-computer.  CIVD  will  also 
be  added  to  the  list  during  the  development  phase. 

Method  of  Instruction.  The  stimulus  medium  and  the  response 
acceptance  mechanism  are  next  used  to  select  a  method  of 
instruction.  This  selection  process  is  basically  one  of 
identifying  training  methods  that  support  the  requirements 
imposed  by  both  the  stimulus  presentation  medium  and  the  response 
acceptance  mechanism,  and  then  selecting  the  optimum  method  or 
methods.  Optimality,  in  this  context,  is  determined  by 
characterizing  both  the  stimulus  presentation  medium  and  the 
response  acceptance  mechanism  in  terms  of  several  additional 
factors  and  then  tallying  the  number  of  "matches”  for  each 
method.  The  method  having  the  largest  number  of  such  matches  is 
selected. 

Factors  used  in  the  TAM  to  further  characterize  the  stimulus 
presentation  medium  are  listed  as  follows: 

•  Pacing  Controller.  The  mechanism  controlling  the  speed 
of  application  for  the  presentation  medium  (program, 
trainee,  or  both) . 

•  Stimulus  Content.  Visual,  verbal,  or  audio. 

•  Next  Learning  Activity.  The  mechanism  controlling  the 
presentation  sequence  of  items  (program,  instructor,  or 
trainee) . 

Response  acceptance  mechanisms  are  further  characterized  in 
terms  of: 

•  Pacing  Controller.  Trainee,  instructor,  or  program 

•  Next  Learning  Activity.  Program,  instructor,  or  trainee 

•  Type  of  Evaluation  Provided.  Individual  versus  group; 
selected  versus  constructed 

•  Feedback.  Immediacy  (immediate  versus  delayed) ;  source 
(intrinsic  versus  extrinsic) 
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Appendix  B,  Figures  B-10  and  B-ll,  contains  the  method 
selection  tables  proposed  for  use  in  the  TAM.  Appendix  B 
provides  a  detailed  example  of  how  stimulus  presentation  medium 
and  the  response  acceptance  mechanism  characteristics  are  used  to 
identify  a  preferred  method  of  instruction.  The  methods  of 
instruction  proposed  for  the  TAM  are  listed  below. 

•  Case  Study 

•  Computer-Aided  Instruction 

•  Demonstration 

•  Games 

•  Group  Interview 

\ 

•  Guided  Discussion 

•  In-Basket  Exercise 

•  Peer  Tutor 

•  Programmed  Instruction 

•  Programmed  Practical  Exercise 

•  Role  Playing 

•  Study  Assignment  Book 

•  Traditional  Classroom 

•  Traditional  Practical  Exercise 

•  Tutoring 

Modifications  to  the  list  above  which  will  be  made  during 
the  development  phase  include  the  addition  of  CIVD  as  a  method  of 
instruction. 

As  a  check  on  the  validity  of  the  instructional  method 
selection  process,  the  user  also  compares  reconmended 
instructional  methods  with  the  learning  algorithm  to  insure  that 
the  two  are  compatible.  Compatibility  is  determined  by  the 
source  and  immediacy  of  feedback  that  can  be  provided  within  the 
context  of  a  particular  learning  algorithm.  Figure  B-12  presents 
the  Learning  Algorithm-Instructional  Method  compatibility  table 
used  in  the  TAM. 

Training  Setting.  The  final  aspect  of  the  training 
prescription  element  involves  specifying  a  training  setting. 
Training  settings  commonly  available  within  a  military 
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environment  are  selected  on  the  basis  of  three  criteria,  listed 
as  follows: 

1.  The  number  of  trainees  likely  to  be  involved  (a  small 
number  versus  five  or  more) . 

2.  The  nature  of  trainee  interactions  (individual  versus 
team) . 

3.  Whether  or  not  equipment  manipulations  are  required. 

Training  settings  currently  employed  in  the  TAM  are  listed  as 
follows: 


•  Small  Group  Site 

•  Large  Group  Site 

•  Individual  Carrel 

•  Small  Group  Carrel 

•  Traditional  Classroom 

Figure  B-13  presents  the  logic  of  the  training  setting 
selection  process. 

In  summary,  the  TAM's  training  prescription  decision  process 
results  in  a  series  ox  training  recommendations  for  each  task 
under  consideration.  The  training  prescription  includes  the 
following  information: 

•  Learning  Algorithm.  A  model  instructional  approach 

•  Stimulus  Presentation  Medium.  The  medium  used  in 
presenting  the  stimulus  content  of  a  task 

•  Response  Acceptance  Mechanism.  The  medium  used  in 
accepting  and  evaluating  trainee  responses  and  in 
providing  feedback  (i.e.,  knowledge  of  results) 

•  Instructional  Method 

•  Instructional  Setting 

Figure  9  provides  an  overview  of  the  complete  training 
prescription  generation  process. 


Training  Priority.  Time,  and  Option 

Element  three  of  the  TAM  is  concerned  with  training 
priority,  training  time,  and  training  option.  Training  priority 
involves  the  level  of  training  to  be  provided  for  a  given  task. 
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Figure  9.  Training  prescription  generation 
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Time  does  not  permit  training  all  tasks  in  a  resident 
instructional  setting.  Training  priority  ratings,  when  combined 
with  training  time  constraints,  are  used  to  determine  which  tasks 
should  be  trained  in  a  resident  setting,  those  that  can  be 
relegated  to  OJT,  and  those  that  require  no  formal  training 
(i.e.,  informal  OJT  is  likely  to  be  sufficient). 

Training  priority  is  determined  on  the  basis  of  user 
responses  to  the  ten  critical  dimensions  shown  in  Table  6.  Based 
on  these  responses,  tasks  are  sorted  into  one  of  three  training 
level  categories: 

1.  Certification.  Proficiency  for  each  and  every  task 
procedural  step  must  be  demonstrated.  These  are  usually  highly 
critical  tasks  such  as  those  performed  in  emergency  situations. 

2.  Qualification.  The  trainee  must  demonstrate  proficiency 
on  a  sample  of  the  procedural  steps  in  a  task. 

3.  Familiarization.  A  demonstration  of  task-related 
activities  is  sufficient. 

Criticality  ratings  are  also  used  for  other  training-related 
purposes.  For  example,  tasks  that  can  be  trained  in  less  time 
during  mobilization  are  identified.  Tasks  that  are  performed 
only  in  wartime  are  identified  for  special  mobilization  training. 
And  finally,  tasks  which  are  based  upon  skills  that  deteriorate 
quickly  when  not  in  use  are  identified  as  requiring  periodic 
refresher  training. 

The  iCwund  uspect  of  element  three  concerns  training  time. 

In  this  regard,  training  specialists  are  asked  to  provide 
estimates  of  the  length  of  time  required  to  reach  the  performance 
criterion  for  each  task,  given  the  instructional  medium  and 
method  used  and  the  Target  Audience  Description.  In  making 
training  time  estimates,  users  should  consider  predecessor  or 
referent  system  training  times,  as  well  as  the  results  of  any 
relevant  training  effectiveness  analyses  (i.e.,  a  post-fielding 
TEA  on  a  predecessor  or  referent  system) .  Both  the  target 
audience  and  predecessor  training  system  information  required  to 
complete  this  analysis  are  contained  in  data  bases  produced  by 
the  training  constraints  module  (TCM) . 

The  selection  of  a  Training  Option — Resident  Instruction, 
OJT,  or  No  Formal  Training — is  relatively  straightforward.  Tasks 
that  require  a  Certification  level  of  training  are  selected  for 
Resident  instruction;  tasks  that  require  Qualification-level 
training  are  selected  for  either  Resident  instruction  or  formal 
OJT;  and  tasks  requiring  only  Familiarization  training  are 
relegated  to  OJT  or  slated  to  receive  no  formal  training. 

Training  time  constraints  that  are  evident  following  the 
completion  of  the  Training  Program  Synthesis  step  (Element  4)  are 
used  to  further  separate  tasks  into  Training  Option  categories. 
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Table  6 


Task  criticality  dimensions 


Task  Criticality  Dimensions 


A.  Learning  Difficulty  -  Is  the  task  hard  to  learn? 

L  -  Easy  to  learn  -  can  be  self-trained 

M  -  Some  difficulty  in  learning  -  requires  some 
assistance  to  learn 

H  -  Hard  to  learn  -  requires  supervision,  extensive 
practice  or  special  equipment  or  environment 

B.  Performance  Difficulty  -  Is  the  task  hard  to  perform? 

L  -  Easy  to  perform  -  can  peform  correctly  on  initial 

effort  and  each  repetition  -  include  only  simple  skills 
M  -  Some  difficulty  in  performing  -  requires  practice 

and  some  supervision  to  perform  -  moderate  level  skills 
H  -  Hard  to  perform  -  additional  practice  and 

supervision  required  for  performance  -  high  probability 
of  some  performance  failures  -  includes  complex  skills 
or  skills  integration 

C.  Time  Delay  Tolerance  -  What  is  the  time  allowed  between 
receiving  the  task  cue  and  starting  the  performance? 

L  -  No  need  to  start  task  at  any  specific  time 

M  -  Task  start  can  be  delayed  for  several  minutes 

H  -  Must  begin  immediately  or  within  a  few  minutes  after  cue 

D.  Consequence  of  Inadequate  Performance  -  How  serious  is 
the  effect  of  improper  performance  or  non-performance 
on  unit  or  individual  missions? 

L  -  Has  little  or  no  effect  on  mission  of  individual  or  unit 
M  -  Could  degrade  or  delay  mission  peformance 
H  -  Could  result  in  mission  failure 

E.  Immediacy  of  performance  -  How  soon  after  arrival  in 
field  unit  could  task  performance  be  required  in  wartime? 

L  -  Not  for  several  months 

M  -  Within  the  first  several  weeks  (4-12  weeks) 

H  -  Within  the  first  one  to  four  weeks 

F.  Civilian  Acquired  Skill  -  What  is  the  probability  the 
task  requires  new  skills  not  generally  available  in  the 
target  population 

L  -  All  skills  are  civilian  acquired  by  the  time  of  initial 
entry 

M  -  Some  c f  the  skills  are  civilian  acquired  by  the  time  of 
initial  entry 

H  -  Few  or  none  of  the  skills  are  civilian  acquired  by  the 
time  of  initial  entry 
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Table  6 


Task  criticality  dimensions  (continued) 


G.  Task  Importance  -  Is  the  task  important  to  the  survival  of 
personnel  and  equipment? 

L  -  Failure  or  non-performance  would  have  little  or  no 
effect  on  survival  of  personnel  or  equipment 
M  -  Failure  or  non-performance  could  endanger  personnel  or 
equipment 

H  -  Task  must  be  performed  for  survival  of  personnel  or 
equipment 

H.  Frequency  of  Performance  -  How  often  is  the  task  called 
for  in  peacetime  operations  and  training? 

L  -  Infrequently  -  less  than  once  per  month 
M  -  Occasionally  -  one  or  two  time  per  month 
H  -  Frequently  -  at  least  once  per  week 

I.  Wartime  Task  -  Is  the  task  oriented  towards  wartime 
operations? 

1  -  Peacetime  only  -  task  is  not  performed  during  wartime 

2  -  War  &  Peace  -  task  can  be  performed  both  in  peace  and  in 

war 

3  -  Wartime  only  -  task  is  never  performed  or  practiced 

until  wartime 

J.  Proficiency  Decay  Rate  -  How  frequently  must  the  task 
be  performed  to  assure  that  skills  are  not  reduced 
below  standards? 

L  -  Task  skills  require  little  or  no  practice  to  retain 
proficiency 

M  -  Task  requires  infrequent  practice  -  once  every  one  to 
three  months 

H  -  Frequent  practice  required  -  more  often  than  once  a 
month 
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In  summary,  the  results  by  task  provided  through  the 
exercise  of  element  four  are: 

•  Training  Level  Recommendation.  Certification, 
qualification,  or  familiarization 

•  A  Training  Time  Estimate.  The  training  time  judged 
necessary  to  meet  the  performance  criterion 

•  Training  Option  Recommendations:  Resident,  OJT,  or  Ho 
Formal  Training 

Figure  10  provides  a  summary  overview  of  the  steps 
comprising  element  three  of  the  Analysis  module. 


Training  Program  Synthesis 

The  fourth  element  comprising  the  TAM  concerns  training 
program  synthesis.  During  this  portion,  users  integrate  the 
results  from  the  previous  elements  to  produce  a  tabular 
description  of  total-system  training  requirements.  This  tabular 
description  can  serve  as  the  basis  for  the  development  of  an 
Outline  Individual  and  Collective  Training  Plan. 

The  basis  for  the  tabular  description  of  training 
requirements  is  the  Training  Task  Matrix,  a  portion  of  which  is 
shown  as  Figure  11.  Dimensions  defining  the  Training  Task  Matrix 
are: 

•  Task  Category: 

Operations 

Maintenance 

Support 

•  Type  of  Training: 

Initial  Skill  Acquisition 

Individual  and  Team  Tactical  Performance  (Unit) 
Combined  Arms  (Multi-Unit) 

-  Skill  Maintenance  (Refresher) 

•  Rank  of  Typical  Task  Performer 

-  E2  -  E4  (first-term  enlisted) 

-  E5  or  above  (career  enlisted) 

-  Officer 

Tasks  are  arrayed  along  dimension  one  by  category 
(operations,  maintenance,  or  support) ;  within  categories,  tasks 
can  also  be  broken  out  by  tentative  job  positions  (i.e.,  MOSs)  if 
such  information  is  available.  Entries  in  the  matrix  then 
consist,  by  task,  of:  (1)  the  training  prescription,  (2) 
training  level,  (3)  training  time,  and  (4)  the  training  option 
(i.e..  Resident,  OJT,  or  No  Formal  Training).  When  the  time 
available  for  training  by  training  type  and  by  tentative  job 
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Figure  10. 


Training  priority,  time,  and  option. 


Figure  11.  Training  task  matrix 


position  (derived  from  the  Constraints  module)  is  compared  with 
aggregate  training  times,  a  more  complete  characterization  of 
training  options  can  be  made.  Recall  that  upon  completing  the 
previous  element,  some  tasks  might  be  recommended  for  either 
Resident  Instruction  or  OJT.  The  total  time  available  for 
resident  training  will  aid  in  resolving  such  ambiguities 
regarding  training  location. 

The  information  provided  in  the  Training  Task  Matrix  can 
serve  as  the  basis  for  the  development  of  a  total-system  training 
concept.  In  addition,  the  information  necessary  to  support 
initial  training  device  concept  development  (i.e.,  an  initial 
TDS)  is  present.  Users  thus  have  much  of  the  information 
required  to  develop  an  OICTP.  These  latter  issues  are,  however, 
beyond  the  requirements  of  the  current  effort. 

At  this  point  in  the  TAM,  the  user  will  have  constructed  a 
general  description  of  an  "optimal"  or  recommended  training 
program  for  the  materiel  system  of  interest.  To  accomplish  the 
goals  of  the  current  SOW,  this  "optimal"  training  system  must  now 
be  modified  based  upon  the  information  developed  by  Training 
Constraints  Model  (TCM) .  This  modified  training  description  will 
represent  the  "most  likely"  training  to  exist  for  the  system. 

The  next  section  describes  the  process  which  will  be  used  to 
derive  the  "most  likely"  training  program. 


Training  Resource  and  Constraint  Assessment 

The  transformation  of  the  description  of  an  "optimal" 
training  program  provided  by  the  fourth  element  to  the  "most 
likely"  training  program  produced  by  the  fifth  element  of  the  TAM 
is  a  five  step  process.  The  principal  steps  in  the  process 
include : 

1.  Modification  of  the  Training  Task  Matrix  by,  eliminating 
methods  of  instruction  or  not  listed  as  feasible  in  output  from 
the  training  technology  constraints  data  base  of  the  TCM. 

2.  Adjustment  of  any  segment  of  the  recommended  training 
program  which  requires  more  resources  in  any  resource  category 
than  the  level  identified  as  available  in  the  training  resources 
constraint  data  base. 

3.  Development  of  a  dynamic  resource  consumption  matrix 
based  on  a  training  outline  formed  from  the  predecessor  training 
system  resource  requirements  matrix  and  the  training  prescription 
produced  by  the  TAM. 

4.  Development  of  a  most  likely  training  outline  with  a 
dynamic  resource  consumption  matrix  that  fits  within  the  training 
resource  constraints  boundaries  identified  in  the  TCM  resource 
constraints  data  base. 
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5.  Modification  of  the  Training  Task  Matrix  existing  after 
step  2  (above)  to  fit  within  the  time  and  resource  boundaries  of 
the  training  outline  developed  in  step  4. 

Each  of  the  five  steps  is  described  in  detail  below. 

Elimination  of  Training  Methods  and  Technology.  The  easiest 
and  most  straightforward  modification  to  be  made  to  the 
recommended  Task  Training  Matrix  will  be  the  elimination  of 
methods  of  instruction  which  the  TCM  identified  as  unlikely  to  be 
available.  After  eliminating  those  methods  of  instruction  which 
the  TCM  identified  as  unavailable,  the  user  will  repeat  the 
instructional  method  selection  process  for  the  affected  tasks  by 
selecting  available  methods  with  the  greatest  number  of  matches 
to  requirements  imposed  by  the  stimulus  presentation  medium  and 
response  acceptance  mechanism  requirements.  A  new  Task  Training 
Matrix  will  be  produced  at  the  end  of  this  step. 

Adjustment  for  Exceeding  Resource  Availability.  The  first 
adjustment  for  exceeding  training  resource  availability  will 
occur  in  the  second  step  of  the  modification  of  the  recommended 
training  program.  This  will  be  a  relatively  straightforward 
comparison  of  the  total  requirements  for  classrooms,  instructors, 
etc.  indicated  by  the  Task  Training  Matrix  with  the  availability 
limits  in  the  training  resource  constraints  data  base.  The 
adjustments  which  might  be  made  may  include  reducing  the  amount 
of  time  devoted  to  resident  training,  increasing  the 
student/instructor  ratio,  shifting  some  training  tasks  to  on-the- 
job  training,  etc. 

Violations  of  available  training  resource  constraints  are 
fairly  easy  to  identify  for  resources  found  in  resident  training 
environments.  Furthermore,  since  resident  instruction  is 
individually  oriented,  adjustments  to  instructional  programs  can 
be  made  with  the  assumption  that  scheduling  of  individuals  to 
attend  courses  can  be  modified  to  some  degree. 

Identification  of  training  resource  constraint  violations 
for  unit  training  are  considerably  more  complex  and  more 
difficult  to  correct.  This  problem  is  particularly  acute  during 
the  time  that  a  new  system  is  being  fielded.  It  is  at  this  time 
that  the  training  system  and  the  units  being  trained  are  all  in  a 
state  of  transition.  The  distribution  plan  for  the  new  system, 
unit  training  resource  consumption  rates,  and  conflicting 
scheduling  for  limited  resources  such  as  gunnery  ranges  must  be 
considered  simultaneously  in  projecting  the  most  likely  training 
'•’hirh  will  actually  occur  in  units  receiving  the  new  materiel 
system.  The  development  of  a  dynamic  resource  consumption  matrix 
is  the  first  step  in  solving  this  problem. 

Development  of  the  Dynamic  Resource  Consumption  Matrix.  One 
of  the  data  bases  developed  in  the  TCM  is  a  predecessor  system 
training  resource  requirements  matrix.  As  described  earlier, 
this  data  base  provides  a  matrix  of  resources  consumed  during 
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each  phase  of  training  for  a  single  training  cycle  with  the 
predecessor  system.  This  matrix  provides  the  starting  point  for 
the  development  of  a  dynamic  resource  consumption  matrix  for  the 
materiel  system  of  interest.  The  development  of  the  dynamic 
resource  consumption  matrix  involves  the  four  steps  listed  below: 

•  Modify  the  predecessor  training  system  outline  to  match 
the  training  outline  of  the  new  system 

•  Determine  the  resource  consumption  rate  for  a  single 
cycle  of  the  new  training  outline 

•  Model  the  total  resource  consumption  for  all  units 
receiving  the  new  materiel  system 

•  Output  the  dynamic  resource  consumption  matrix 

The  modification  of  the  outline  of  the  predecessor  training 
system  to  match  as  closely  as  possible  the  anticipated  training 
outline  of  the  new  system  concept  is  based  on  known  or  expected 
differences  between  the  systems.  Of  most  importance  are  changes 
that  directly  affect  the  need  for  training  resources.  This 
process  will  be  somewhat  subjective  and  will  require  subject 
matter  expertise  and  familiarity  with  the  predecessor  materiel 
system  and  the  new  system  concept.  The  important  factor  is  to 
carefully  document  all  decisions  and  record  the  logic  used  to 
modify  resource  demands.  Figure  12  illustrates  a  simple 
collective  training  outline  for  an  aviation  system. 

Once  a  new  system  training  outline  has  been  determined  and 
the  accompanying  training  resource  consumption  rate  for  a  single 
cycle  of  training  specified,  the  next  step  is  to  model  total 
resource  consumption  over  time  for  all  units  receiving  the  new 
materiel  system.  The  actual  mathematical  model  used  to  calculate 
resource  consumption  is  a  four  dimensional  model.  A  discussion 
of  the  structure  and  function  of  the  model  is  beyond  the  scope  of 
the  current  effort.  The  essential  point  to  be  made  is  that  the 
model  makes  it  possible  to  examine  the  training  schedule  and 
resource  consumption  (by  type  of  training  resources)  for  all 
units  training  with  the  new  system.  The  model  provides 
information  regarding  aggregate  resource  consumption  for  each 
fiscal  year  and  allows  one  to  identify  total  resource  demand  (by 
critical  training  resource  category)  at  any  point  in  time.  This 
model  produces  the  output  referred  to  as  the  dynamic  resource 
consumption  matrix.  This  matrix  is  most  easily  examined  in  a 
graphic  form.  Figure  13  presents  a  sample  printout  of  the 
resource  consumption  for  one  class  of  training  resources  over  a 
single  year. 

Development  of  a  Most  Likely  Training  Outline.  The  dynamic 
resource  consumption  matrix  output  will  identify  violations  of 
resource  constraints  developed  in  the  TCM.  More  importantly,  the 
model  also  allows  one  to  adjust  the  training  outline  for  unit 
training  to  test  alternative  training  programs  which  result  in 
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Figure  13.  Dynamic  resource  consumption  matrix. 


resource  consumptions  below  projected  limits.  Examples  of 
adjustments  to  the  training  outlines  include  extending  or 
shortening  times  for  particular  segments  of  training,  using 
simulators  instead  of  live  fire  ranaes,  etc.  The  efficient 
generation  of  such  alternatives  requires  familiarity  with  the 
materiel  system  concept  and  unit  training  processes.  However, 
the  model  provides  very  rapid  feedback  on  the  impact  of  such 
changes  on  resource  consumption. 

The  model  calculates  other  measures  in  addition  to  resource 
consumption.  These  measures  of  the  relative  effectiveness  of  the 
training  outline  alternatives  include  the  average  length  of  time 
required  to  complete  a  training  cycle  and  the  average  "down"  time 
for  each  unit. 

Through  an  iterative  process  of  testing  alternatives,  the 
user  will  develop  a  training  outline  of  the  collective  training 
which  consumes  resources  at  a  rate  within  the  limits  established 
by  the  TCM.  This  training  outline  with  its  associated  resource 
consumption  rate  is  considered  the  "most  likely"  structure  for 
the  collective  training  for  the  system. 

Modification  of  the  Task  Training  Matrix.  The  "most  likely" 
training  outline  produced  through  the  modeling  of  dynamic 
resource  consumption  described  in  the  previous  step  must  now  be 
compared  to  the  collective  training  tasks  in  the  Task  Training 
Matrix.  Modifications  in  the  training  time,  methods,  location, 
and  other  factors  must  be  made  to  the  matrix  to  "fit"  the 
training  outline  developed  in  the  previous  step.  This  modified 
Task  Training  Matrix  will  now  represent  the  TEM's  estimate  of 
"most  likely  training." 


TAM  Output 

The  output  from  the  analysis  module  includes  all  of  the 
items  required  under  TRADOC  Regulation  350-4.  This  includes  a 
consideration  of  the  following: 

•  What  is  to  be  trained:  (Task  Statement)  - 

-  Action  Verb 

-  System  Equipment 
Support  Equipment 
Standards 

-  Conditions 

•  How  training  will  be  done:  (Learning  Prescription)  - 

Learning  Algorithm 
Stimulus  Medium 

-  Response  Acceptance  Mechanism 
Method  of  Instruction 
Learning  Setting 
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•  Where  training  will  be  done:  (Training  Option(s))  - 

-  Resident  Instruction 

-  OJT 

-  No  Formal  Training 

•  When  training  will  be  done 

-  Decisions  pertinent  to  when  training  should  be 
conducted  can  be  made  on  the  basis  of  information 
contained  in  the  Training  Task  Matrix — rank  of  task 
performer  crossed  with  type  of  training  in  question. 

Used  in  combination  with  the  Constraints  module,  the  TAM 
also  provides  users  with  a  quick  determination  of  the  type  of 
training,  by  task,  likely  to  be  provided  for  an  emerging  system. 
Thus,  the  TEM  will  identify  both  an  ••optimal"  Task  Training 
Matrix  and  a  "most  likely"  Task  Training  Matrix.  Those  areas  in 
which  there  are  major  discrepancies  between  the  two  training 
descriptions  are  areas  which  should  be  of  major  concern  to  the 
material  system  designers  as  well  as  the  training  system 
designers.  Thus,  the  exercise  of  the  two  modules  will  provide 
Army  training  developers  with  much  of  the  information  necessary 
to  define  an  appropriate  training  concept,  identify  potential 
constraints,  estimate  the  impact  of  the  constraints  on  the  nature 
of  the  projected  training  system,  and  identify  potential  training 
performance  deficiencies  resulting  from  the  training  constraints. 


The  Feedback  and  Interface  Module  fTIM) 


The  final  module  of  the  TEM  is  the  Feedback  and  Interface 
Module  (FIM) .  The  FIM  serves  as  the  output  generator  and  storage 
facility  of  the  TEM.  The  FIM  provides  the  TEM  with  report 
generation  capabilities  to  communicate  the  results  of  exercising 
the  TCM  and  TAM.  The  FIM  will  also  provide  the  user  with  a 
standard  query  capability  for  any  of  the  data  bases  developed 
within  the  TCM  or  TAM. 

The  exact  nature  of  the  reports  generated  by  the  FIM  will  be 
determined  through  research  conducted  with  the  potential  TEM 
users.  The  report  formats  and  content  will  be  tailored  to  meet 
the  users'  needs  to  provide  training-related  information  for  the 
PTEA  and  CTEAs  and  other  system  acquisition  related  documents. 
This  will  include  providing  information  to  support  the  training 
concept,  training  device  concept,  inputs  to  the  ROC,  and  inputs 
the  materiel  system  RFP. 

In  addition  to  meeting  the  needs  of  the  training  developers, 
the  FIM  will  provide  valuable  data  to  the  materiel  system 
designers.  As  noted  earlier,  in  addition  to  the  information 
describing  the  "most  likely"  training  for  each  task  identified  by 
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Product  1,  the  FIM  will  generate  reports  related  to  potential 
training  problem  areas  and  predecessor  system  training  problems 
which  may  have  been  related  to  interface  design  problems. 

The  FIM  will  be  developed  from  standard,  commercially 
available  software. 


Development  of  the  TEM 


The  authors  have  already  noted  that  the  major  factors  which 
we  feel  must  be  considered  in  the  design  and  development  of  the 
TEM  are  the  needs  of  the  user  and  the  user's  environment.  The 
plan  for  the  development  of  the  TEM  reflects  this  factor  as  the 
primary  "high  driver."  As  currently  structured,  the  current 
effort  has  two  remaining  phases,  a  product  design  and  a  product 
development  phase. 

The  key  to  successful  development  of  Product  4  will  be  in 
designing  the  product  to  ensure  user  acceptance  and 
institutionalization  of  the  methodology.  For  this  reason  the 
next  action  we  recommend  in  design  of  Product  4  is  an  in-depth 
analysis  of  user  needs  and  the  likely  hardware  and  software 
systems  which  will  exist  when  Product  4  is  ready  for  distribution 
to  users.  As  noted  earlier,  we  believe  that  the  members  of  the 
DOTD-NSTO  will  be  the  primary  users.  The  next  step  in 
development  of  Product  4  will  be  to  establish  the  information 
requirements,  characteristics,  and  user  biases  which  exist  for 
this  target  audience.  This  information  is  particularly  critical 
for  the  development  of  the  Feedback  and  Interface  Module  (FIM) . 

It  is  through  the  user  needs  analysis  that  we  will  establish  the 
most  appropriate  content  and  format  to  be  included  in  reports 
generate  by  the  TEM. 

There  should  be  user  involvement  in  the  remainder  of  the 
design  and  development  process  to  aid  in  acceptance  and 
institutionalization  of  the  product.  Members  of  the  user 
community  should  be  briefed  on  the  concept  for  Product  4  and 
provided  with  the  opportunity  to  make  recommendations  for 
modifications  of  the  type  of  information  output  they  require  and 
the  best  way  to  provide  training  on  the  product.  Establishing 
the  degree  of  computer  literacy  in  the  user  population  will  be  a 
critical  factor  in  aiding  the  development  of  a  product  that  gains 
user  acceptance. 

An  additional  aspect  of  the  user  needs  analysis  will  be  to 
identify  means  through  which  the  TEM  might  be  institutionalized 
within  the  user  community.  This  includes  identification  of 
formal  training  settings  (schools  and  courses)  in  which  the  users 
previously  obtained  information  related  to  training  analysis. 
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Additional  data  related  to  this  topic  will  include  an  assessment 
of  the  viability  of  CBT  embedded  in  the  TEM  as  a  means  for 
training  the  users. 

A  second  type  of  information  required  for  development  of 
Product  4  is  identification  of  the  hardware  and  software  ^systems 
currently  used  in  DOTD  offices  at  various  combat  development 
centers.  We  currently  plan  to  provide  the  finished  TEM  for  an 
I**M  compatible  system. 

•  In  developing  the  prototype  for  Product  4,  HTI  will  ensure 
that  the  tool's  architecture  is  comfortable  to  TRADOC's  emerging 
Schoolhouse  concept,  denoted  the  Training  Support  System  (TSS) 

The  TEM  may  be  incorporated  into  the  TSS  or  it  may  function  best 
as  a  stand-alone  methodology  for  use  with  more  powerful  micro¬ 
computers. 

Initial  prototypes  for  major  portions  of  the  TEM  are  already 
in  existence.  The  majority  of  the  TAM  has  been  programmed  on  a 
mainframe  computer  and  tested  in  several  experimental  studies. 

The  structure  of  the  model  used  to  calculate  the  dynamic  resource 
consumption  matrix  has  also  been  completed.  The  prototype  of  the 
resource  consumption  model  has  been  used  successfully  in  training 
analyses  conducted  during  the  concept  development  phase  of  a 
major  system.  The  model  currently  resides  on  a  micro-computer. 

Thus,  the  major  model  prototyping  which  must  be  completed 
for  development  of  the  TEM  lies  in  the  development  of  prototypes 
of  the  major  elements  of  the  TCM.  Since  the  TCM  is  essentially  a 
data  base  development  module,  rhe  programming  required  tu 
implement  these  portions  of  the  model  is  fairly  routine.  In 
fact,  as  part  of  the  prototyping  conducted  for  Product  2,  HTI 
has  successfully  programmed  routines  which  allow  the  user  to 
dimensionalize  spreadsheets  using  a  menu  driven  system  requiring 
no  working  knowledge  of  spreadsheet  development.  The  user  must 
only  respond  to  prompts  by  indicating  the  number  of  categories 
and  corresponding  labels  for  each  dimension  of  the  model.  The 
programming  required  for  all  of  the  TCM  components  can  probably 
be  accomplished  using  standard  data  base  programming  packages 
such  as  dBase  111+. 

The  most  significant  developmental  task  for  the  TEM  besides 
ensuring  user  acceptance  is  the  development  of  the  user  guidance 
for  obtaining  constraints  data  included  in  the  data  base  programs 
of  the  TCM.  This  will  require  continued  research  and 
interactions  with  users  to  identify  the  most  helpful  documents 
from  which  to  derive  training  resource  constraints  data.  We  must 
also  identify  the  best  sources  for  obtaining  such  documents  or 
information.  As  noted  earlier,  now  that  we  have  identified  the 
major  sources  of  the  information,  the  developmental  task  at  hand 
is  to  provide  more  specific  guidance  as  to  where  and  how 
information  can  be  obtained  in  an  efficient  manner.  This  will 
require  analysis  of  various  types  of  training  documentation 
produced  and  maintained  by  agencies  such  as  the  proponent  combat 
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development  centers,  PM-TRADE,  selected  elements  of  the  DA  Staff, 
and  selected  elements  of  TRADOC,  FORSCOM,  and  other  MACOM 
headquarters.  If  at  all  possible,  the  TCM  data  base  programs 
will  be  designed  to  allow  electronic  transfer  of  training 
resource  data  from  existing  computer  data  bases  such  as  the  AETIS 
and  AIMS. 


Institutionalization  of  Product  4 


While  considerable  time  has  already  been  devoted  to 
discussion  of  the  design  and  development  steps  taken  to  aid  in 
gaining  acceptance  and  institutionalization  of  the  TEM,  it  is 
important  to  note  the  steps  required  in  implementation  of  the 
TEM.  The  authors  believe  that  institutionalization  of  Product  4 
will  occur  only  if  TRADOC  identifies  an  internal  proponent  for 
the  product.  The  product  must  become  an  accepted  part  of  the 
training  analysis  portion  of  system  acquisition  process.  The  use 
of  the  TEM  must  be  part  of  the  formal  education  and  training 
system  for  training  developers  and  TRADOC  must  support  the 
product  with  necessary  documentation  and  software  modifications 
to  improve  the  product  as  new  information  becomes  available  and 
the  user  community  gains  experience  with  the  product. 

If  TRADOC  assumes  this  proponent  role  and  the  TEM  has  been 
successfully  designed  to  meet  the  needs  of  the  user  audience,  the 
product  will  be  institutionalized.  If  either  ingredient  is 
missing,  however,  Product  4  will  join  the  ranks  of  other 
discarded  models  and  methods  to  automate  or  formalize  the 
training  analysis  process.  While  HTI  has  attempted  to  ensure 
that  the  TEM  meets  the  needs  of  the  DOTD  users,  the 
responsibility  for  acquiring  the  TRADOC  blessing  and  proponency 
of  Product  4  or  any  of  the  other  MPT  Estimation  Products  will 
rest  primarily  with  the  Army  Research  Institute. 


User  Training 

The  TEM  is  designed  to  be  as  user  friendly  and  self-guiding 
as  possible.  As  was  described  earlier,  the  automated  portions  of 
the  methodology  are  menu  driven  and  prompt  the  user  for  required 
input.  The  online  user  prompts  and  help  functions  will  be 
supported  by  documentation  which  clearly  outlines  all  functions 
(both  automated  and  manual)  required  of  the  user. 

Ideally,  the  users  would  receive  some  formal  training  in  a 
classroom  environment  to  at  least  familiarize  them  with  the 
concepts  and  procedures  involved  in  using  the  TEM.  This  is 
viewed  as  unlikely,  however,  and  the  user  prompts,  on  line  help, 
software  documentation  and  overview  training  manual  accompanying 
the  TEM  will  be  pilot  tested  with  a  sample  of  users  to  ensure 
that  they  provide  adequate  training  for  the  product. 
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Person  Hours  Required  to  Use  the  Product 


One  of  the  goals  for  Product  4  was  to  develop  a  relatively 
quick  method  for  estimating  the  most  probable  training.  As  we 
developed  the  TEM  and  worked  another  early-on  training  analysis 
for  a  major  Army  system  it  became  apparent  that  the  problem  of 
developing  a  useful  estimate  of  most  likely  training  is  a  complex 
task.  On  the  other  hand,  we  have  also  learned  that  it  is  not  the 
entry  or  analysis  of  the  data  which  is  time  consuming,  but 
rather,  it  is  the  collection  of  the  data.  This  is  why  the 
development  of  the  TEM  will  include  a  major  focus  on  development 
of  on-line  guidance  to  help  users  identify  data  sources  and  to 
guide  extraction  of  data  from  existing  documentation.  This 
aspect  of  TEM  should  provide  a  major  time  savings  for  users  of 
the  product. 

The  estimate  for  total  person-hours  required  to  exercise  the 
TEM  for  analysis  of  a  comprehensive  training  program  on  a  major 
acquisition  is  approximately  560  person-hours.  This  estimate  is 
based  on  our  experience  with  the  pilot  testing  of  the  TDDA  which 
is  the  predecessor  to  TAM  and  research  we  have  conducted  using 
the  dynamic  training  resource  consumption  model  which  requires 
the  collection  of  much  of  the  data  used  in  the  TCM  of  the  TEM. 

The  estimate  may  increase  or  decrease  by  approximately  25% 
depending  on  the  degree  to  which  the  user  possess  the  documents 
and  data  required  to  derive  training  constraints  information.  If 
the  user  has  a  reasonable  collection  of  predecessor  training 
system  documentation  and  a  curxent  inventory  of  relevant  training 
resources  from  the  proponent  schools  and  units  receiving  the  new 
units,  the  person  hours  are  likely  to  be  reduced  by  25%.  If  the 
user  is  starting  from  scratch  on  collection  of  data  sources,  the 
person  hours  expended  will  probably  increase  by  25%  or  more. 
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AFPENDIX  A 


TASK  VERB-LEARNING  ALGORITHM  LISTING 


VERB 

ALGORITHM 

Abide  by 

Attitudes 

Accepting 

Attitudes 

Accommodate 

Problem  Solving 

Acquire 

Steering  &  Guiding 

Continuous  Movement 

Activate 

Motor  Chaining 

Adapt 

Problem  Solving 

Adjust 

Motor  Chaining 

Adjust  to 

Problem  Solving 

Advise 

Communicating 

Aim 

Steering  &  Guiding 

Continuous  Movement 

Align 

Motor  Chaining 

Allocate 

Classifying 

Analyze 

Rule  Using 

Answer 

Communicating 

Anticipate 

Rule  Using 

APFly 

Rule  Using 

Arrange 

Classifying 

Assemble  (Dis) 

Motor  Chaining 

Assign 

Classifying 
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VERB 


AL22E11M 


Associate 

Identifying  Symbols 

Attend 

Monitoring 

Calculate 

Classifying 

Catalogue 

Classifying 

Categorize 

Classifying 

Characterize 

Classifying 

Check 

Rule  Using 

Choose 

Decision  Making 

Cite 

Verbal  Chaining 

Classify 

Classifying 

Clean 

Motor  Chaining 

Close-Open 

Motor  Chaining 

Collect 

Classifying 

Compare 

Identifying  Symbols 

Compensate 

Steering  &  Guiding 

Continuous  Movement 

Compile 

Classifying 

Comply  with 

Attitudes 

Compose 

Problem  Solving 

Compute 

Rule  Using 

Conclude 

Rule  Using 

Conform  to 

Attitudes 
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ALGORITHM 


VERB 

Connect  (Dis) 

Construct 

Contrast 

Contrive 

Control 

Converse 

Convert 

Coordinate 

Copy 

Correlate 

Create 

Cut 

Deduce 

Demonstrate 

Design 

Detect 

Determine 

Develop 

Devise 

Diagnose 

Diagram 

Di  f  f  erer.tiate 

Direct 


Motor  Chaining 
Problem  Solving 
Identifying  Symbols 
Problem  Solving 
Steering  4  Guiding 
Continuous  Movement 
Communicating 
Rule  Using 
Rule  Using 
Motor  Chaining 
Decision  Making 
Problem  Solving 
Gross  Motor  Skills 
Rule  Using 
Rule  Using 
Problem  Solving 
Monitoring 
Decision  Making 
Problem  Solving 
Problem  Solving 
Decision  Making 
Rule  Using 
Identifying  Symbols 
Communicating 
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YERB 


ALGORITHM 


Discern 

Discover 

Distinguish 

Divide 

Drrft 

Draw 

Drive 

Energize  (De) 

Enumerate 

Equate 

Estimate 

Evaluate 

Examine 

Explain 

Express 

Extrapolate 

Figure 

File 

Fly 

Generalize 

Grade 

Grasp 

Group 


Identifying  Symbols 
Problem  Solving 
Identifying  Symbols 
Classifying 
Gross  Motor  Skills 
Gross  Motor  Skills 
Steering  6  Guiding 
Continuous  Movement 
Motor  Chaining 
Verbal  Chaining 
Rule  Using 
Rule  Using 
Rule  Using 
Rule  Using 
Rule  Using 
Communicating 
Rule  Using 
Rule  Using 
Classifying 
Steering  &  Guiding 
Continuous  Movement 
Rule  Using 
Classifying 
Gross  Motor  Skills 
Classifying 
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VERB 


Guide 

Steering  k  Guiding 

Continuous  Movement 

Identify 

Identify  Symbols 

Illustrate 

Rule  Using 

Index 

Classifying 

Indicate 

.  Identify  Symbols 

Infer 

Rule  Using 

Inform 

Communicating 

Inspect 

Motor  Chaining 

Install 

Motor  Chaining 

Instruct 

Communicating 

Interpolate 

Rule  Using 

Interpret 

Rule  Using 

Interview 

Communicating 

Invent 

Problem  Solving 

Inventory 

Classifying 

Isolate 

Decision  Making 

Itemize 

Verbal  Chaining 

Judge 

Classifying 

Label 

Identifying  Symbols 

Lead 

Steering  &  Guiding 

Continuous  Movement 
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VERB 

ALGORITHM 

Lift 

Gross  Motor  Skills 

List 

Verbal  Chaining 

.Listen 

Communicating 

Listen  for 

Monitoring 

Load 

Motor  Chaining 

Locate 

Identifying  Symbols 

Look  for 

Monitoring 

Loosen 

Gross  Motor  Skills 

Lubricate 

Motor  Chaining 

Maintain 

Motor  Chaining 

Maneuver 

Steering  &  Guiding 

Continuous  Movement 

Manipulate 

Steering  &  Guiding 

Continuous  Movement 

March 

Gross  Motor  Skills 

Mate 

Identifying  Symbols 

Match 

Identifying  Symbols 

Mix 

Gross  Motor  Skills 

Monitor 

Monitoring 

Name 

Identifying  Symbols 

Navigate 

Steering  &  Guiding 

Continuous  Movement 

Observe 

Monitoring 
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ALGORITHM 

Operate 

Motor  Chaining 

Order 

Communicating 

Organize 

*>ule  Using 

Pick 

Identifying  Symbols 

Pick  up 

Gross  Motor  Skills 

Pilot 

Steering  &  Guiding 

Continuous  Movement 

Plan 

Rule  Using 

Predict 

Rule  Using 

Prepare 

Rule  Using 

Prescribe 

Rule  Using 

Press 

Gross  Motor  Skills 

Program 

Rule  Using 

Project 

Rule  Using 

Pull 

Gross  Motor  Skills 

Push 

Gross  Motor  Skills 

Quote 

Verbal  Chaining 

Raise 

Motor  Chaining 

Rank 

Classifying 

Rate 

Classifying 

Reason 

Problem  solving 

Recall 

Verbal  Chaining 

Recognize 

Identifying  Symbols 

Regulate 

Steering  &  Guiding 

Continuous  Movement 
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VERB 


Reiterate 

Verbal  Chaining 

Relate 

Verbal  Chaining 

Remove/Replace 

Motor  Chaining 

Repeat 

Verbal  Chaining 

Report 

Communicating 

Resist 

Rule  Using 

Resolve 

Problem  Solving 

Respond 

Identifying  Symbols 

Restate 

Verbal  Chaining 

Rotate 

Gross  Motor  Skills 

Run 

Gross  Motor  Skills 

* 

Schedule 

Rule  Using 

Select 

Decision  Making 

Service 

Motor  Chaining 

Set 

Motor  Chaining 

Set  up 

Motor  Chaining 

Sew 

Gross  Motor  Skills 

Sharpen 

Gross  Motor  Skills 

Signal 

Gross  Motor  Skills 

Sing 

Communicating 

Slide 

Gross  Motor  Skills 

Solve 

Rule  Using 

Sort 

Classifying 

Speak 

Communicating 

Specify 
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VERP 


Splice 

Steer 

*  Stencil 
Study 
Stow 
Swim 

Synthesize 

Testify 

Think  Through 

Tighten 

Trace 

Track 

Transcribe 

Translate 

Troubleshoot 

Tune 

Turn  On/Off 

Twist 

Verify 

Wait 

Watch 

Weld 

Write 


Gross  Motor  Skills 
Steering  &  Guiding 
Continuous  Movement 
Gross  Motor  Skills 
Problem  Solving 
Motor  Chaining 
Gross  Motor  Skills 
Problem  Solving 
Communicating 
Problem  Solving 
Gross  Motor  Skills 
Gross  Motor  Skills 
Steering  &  Guiding 
Continuous  Movement 
Rule  Using 
Rule  Using 
Decision  Making 
Motor  Chaining 
Motor  Chaining 
Gross  Motor  Skills 
Rule  Using 
Monitoring 
Monitoring 
Motor  Chaining 
Motor  Chaining 
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APPENDIX  B 


TRAINING  MEDIA/METHOD  SELECTION  PROCESS 


E2-B-1 


E2-B-2 


Figure  B-l.  Verbal  stimuli. 


Figure  B-6.  Equipment 


Voice  response 
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Figure  B-10.  Method  selection 
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E2-B-13 


Figure  B-ll.  Method  selection 
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Method  selection  (continued) 
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Figure  B-12.  Algorithm-method  compatibility  table 
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PRODUCT  * :  TRAINING  CONSTRAINTS  ESTIMATION  AID 

INTRODUCTION 


Objective  of  Paper 

This  concept  paper  describes  an  aid  for  estimating  training 
time  constraints  for  Army  weapon  systems.  The  Training  Con¬ 
straints  Estimation  Aid  (TCEA)  is  one  of  six  automated  products 
being  developed  in  the  Army  Research  Institute's  ( ARI )  Manpower , 
Personnel/  and  Training  aids  for  the  MANPRINT  integration  (MPT)2 
project . 

The  concept  paper  presents  requirements  for  this  aid,  and 
includes  a  detailed  description  of  the  aid’s  steps,  automated 
components,  and  techniques  for  developing  them.  The  concept 
paper  also  outlines  an  approach  for  implementing  the  aid. 


Organization  of  Paper 

This  concept  paper  has  seven  sections.  The  section  of 
product  requirements  includes  the  product's  objectives, 
significant  output,  users,  role  in  the  acquisition  process, 
assumptions,  and  high-level  functional  requirements  or 
constraints . 

The  product  overview  is  a  brief  introduction  to  the  product 
itself,  including  its  output,  steps  the  user  will  go  through  to 
apply  the  product,  our  product  development  approach,  and  the 
anticipated  architecture  of  automated  components.  The  section  on 
the  TCEA ' s  steps  explains  the  input,  output,  process,  user  inter¬ 
face,  and  development  approach  for  each  step  in  great  detail. 

The  next  section  describes  the  TCEA ' s  automated  components  in 
detail.  The  last  two  sections  contain  technology  transfer  issues 
and  the  references,  respectively. 

This  concept  paper  also  includes  several  appendices.  The 
Software  Development  Methodology  is  presented  in  Appendix  A. 
Appendix  B  includes  formats  for  Army  Requirements  Documents. 
Appendix  C  describes  the  process  for  assessing  the  unit  training 
resource  requirements  and  Training  Management  and  Control  System 
(TMACS),  an  automated  system  designed  to  assist  Army  analysts  in 
this  process. 

We  have  purposely  repeated  material  in  Sections  4  and  5.  We 
hope  that  this  will  make  it  easier  to  read  each  section 
independen tly . 
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PRODUCT  REQUIREMENTS 


Output 

The  TCEA  will  produce  two  related  but  distinctively  differ¬ 
ent  outputs.  First,  it  will  produce  estimates  of  the  maximum 
amount  of  time  that  will  be  available  to  train  the  new  system  in 
both  initial  institutional  and  unit  training.  Second,  it  will 
generate  a  set  of  operator  and  maintainer  tasks  for  the  new 
system  and  produce  an  estimate  of  the  likely  amount  of  time  that 
will  be  spent  in  training  these  tasks.  The  estimates  of  maximum 
training  time  will  be  listed  as  constraints  in  the  system  speci¬ 
fication.  Contractors  will  be  required  to  design  a  system  that 
does  not  require  training  which  exceeds  these  maximum  con¬ 
straints.  The  estimates  of  likely  training  time  will  also  be 
provided  to  contractors.  But  they  will  be  described  as  guide¬ 
lines  which  the  contractor  can  use  in  estimating  the  personnel 
characteristics  requirements  associated  with  their  design.  The 
contractors  will  be  free  to  substitute  other  training  times  if 
they  can  provide  a  rationale  for  their  estimate  and  their  total 
training  time  does  not  exceed  the  maximum  training  time 
constraint . 


Role  of  Product  Output  in  Acquisition  Process 

The  TCEA  output  described  above  will  feed  into  several  key 
acquisition  documents.  This  section  identifies  and  describes 
these  documents.  There  are  two  basic  types  of  documents  that 
describe  training  constraints:  Army  requirements  documents  which 
are  designed  to  guide  the  Army  organizations  in  charge  of  de¬ 
veloping  the  system,  and  contractor  specifications  which  are 
designed  to  provide  detailed  guidance  to  the  contractor  develop¬ 
ing  the  system.  Both  types  of  documents  are  closely  related.  In 
fact,  the  contractor  specifications  are  derived  from  the  Army 
requirements  documents. 

Army  Requirements  Documents 

The  Training  Constraints  Estimation  Aid  (TCEA)  will  provide 
input  into  four  Army  requirements  documents:  the  Justification 
for  Major  System  New  Start  (JMSNS),  the  Operations  and  Organiza¬ 
tional  (OiO)  Plan,  the  Letter  of  Agreement  (LOA),  and  the  Re¬ 
quired  Operational  Capability  (ROC).  The  JMSNS,  O&O  Plan,  and 
LOA  should  be  produced  during  the  Requirements/Technology  Base 
Activities  Phase  of  the  Materiel  Acquisition  Phase  (MAP).  The 
ROC  should  be  produced  during  the  Proof-of-Pr inciple  Phase. 
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The  requirements  documents  described  above  (JMSNS,  O&O  Plan, 
LOA,  and  ROC)  are  typically  prepared  by  the  Directorate  of  Combat: 
"evelopment  (DCD)  within  the  proponent  TRADOC  schools  in  close 
coordination  with  the  Army  Materiel  Command  (AMC)  proponent. 

Appendix  B  contains  specific  formats  for  these  requirements 
documents.  The  sections  below  list  details  on  how  training 
constraints  are  presented  in  the  requirements  documents. 

Organizational  and  Operational  Plan.  The  Operational  and 
Organizational  (O&O)  Plan  should  be  developed  during  the 
Requirements  Technology  Base  Activities  Phase  of  the  MAP.  This 
plan  describes  how  the  new  system  will  be  integrated  into  the 
force  structure,  deployed,  operated,  and  supported  in  both 
peacetime  and  wartime. 

Training  issues  should  be  discussed  in  Section  VI  of  the  O&O 

Plan . 


".  .  .VI  Training  Impact  -  Design  of  the  equip¬ 
ment  should  consider  type  and  extent  of  training 
required.  (When  tne  system  is  decided  on,  discuss 
the  type  and  amount  of  training  required  and  the 
need  for  training  devices  and  simulators.)  This 
plan  will  support  preparation  of  the  Training 
Support  Plan.  ( DARCOM/TRADOC  PAM  70-2) 

AR  71-9  and  DARCOM/TRADOC  PAM  70-2,  Chapter  3  provide 
guidance  for  preparation  of  the  O&O  plan. 

Justification  for  Major  System  New  Start.  The  Justification 
for  Major  System  New  Start  (JMSNS)  is  required  when  the  estimated 
cost  to  meet  a  mission  need  exceeds  specified  limits,  or  when 
other  factors  demand  a  DoD-level  review.  Section  F  requires  a 
description  of  "key  boundary  conditions  for  satisfying  the  need, 
such  as  survivability,  logistics,  manpower  and  personnel  con¬ 
straints  in  both  quantity  and  quality;  ..."  (TRADOC  PAM  70-2, 
p.  4.10).  No  explicit  mention  of  training  is  listed  in  the  JMSNS 
format . 

Letter  of  Agreement.  The  Letter  of  Agreement  (LOA)  defines 
the  proposed  system  concept  and  the  research  needed  to  develop 
and  validate  it.  Paragraph  8  of  the  LOA  requires  a  training 
assessment . 

".  .  .  8.  TRAINING  ASSESSMENT.  Discuss  the 
planned  or  system  training  device.  When  required 
include  description  as  an  annex.  New  Equipment 
Training,  operator  and  maintenance  personnel 
training,  and  technical  manuals  and  training 
material  requirements  will  be  stated  in  terms  of 
needs  for  both  the  institution  and  unit  training 
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levels.  The  training  support  plan  will  be 
available  for  evaluation  during  Operational  Test 
( OT }  I  .  .  .  " 

Required  Operational  Capability.  The  Required  Operational 
Capability  (ROC)  is  a  formal  requirements  document  that  "states 
the  minimum  essential  .  .  .  information  necessary  to  initiate  the 
Full-Scale  Development  Phase  or  procurement  of  a  materiel 
system."  ( DARCOM/TRADOC  PAM  70-2,  p.  6.1).  It  addresses  many  of 
the  same  manpower,  personnel,  and  training  issues  the  LOA 
addresses,  but  at  a  lower  level  of  detail. 

The  format  for  Paragraph  8,  TRAINING  ASSESSMENT,  in  the  ROC 
is  almost  identical  to  Paragraph  8  in  the  LOA  (see  above). 
However,  the  ROC  includes  an  additional  requirement  that  the 
training  support  package  be  tested  prior  to  operational  test  (OT) 
II. 


In  summary,  current  Army  requirements  require  an  early 
assessment  of  training  issues.  To  accomplish  this,  an  estimate 
of  the  likely  training  program  for  the  new  system  must  be 
identified  early  in  the  design  process.  None  of  the  Army 
requirements  documents  specifically  requires  an  assessment  of 
training  constraints. 


Documents  for  Presenting  Requirements  to  Contractors 

The  Army  requirements  documents  described  above  define 
system  performance  requirements  for  Army  organizations.  However, 
these  documents  are  typically  not  the  primary  mechanism  for 
presenting  requirements  and  constraints  to  contractors.  The  Army 
requirements  documents  may  be  included  in  the  RFP  package  as 
background  information,  but  the  contractor  is  not  contractually 
bound  to  meet  the  requirements  in  these  documents.  Rather,  the 
contractor  must  adhere  to  the  requirements  cited  in  the  system 
specification . 

MIL-STD-490,  completely  revised  in  1985,  contains  procedures 
for  describing  system  specifications.  This  standard  describes 
system  specifications  in  increasing  detail  as  the  weapon  system 
progresses  through  the  MAP.  The  first  system  specification  _  be 
developed  for  a  major  weapon  is  the  System/Segment  Specification 
(SSS)  or  Type  A  Specification.  The  SSS  should  be  initially 
developed  during  the  Requirements/Technology  Base  Activities 
Phase  of  the  MAP  but  may  be  updated  in  the  subsequent  phase.  It 
is  typically  developed  by  the  combat  developer  within  the 
proponent  school  but  may  be  contracted  out.  Data  Item 
Description  (DID)  DI-CMAN-80008  describes  the  SSS  format. 
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The  SSS  DID  requires  a  description  of  "training  time  and 
locations  available  for  an  effective  training  program."  Table  1 
lists  the  training  requirements  section  of  that  DID. 

The  SSS  also  includes  a  section  on  qualification  require¬ 
ments.  This  section  describes  the  methods  for  demonstrating  that 
each  system  requirement  and  constraint  has  been  met. 

During  later  phases  of  the  MAP,  more  detailed  system  speci¬ 
fications  are  developed  (see  MIL-STD-490 ) .  However,  because 
these  specifications  allocate  functions  among  particular  system 
steps,  they  actually  describe  requirements  at  the  component 
level.  Consequently,  these  specifications  are  not  relevant  to 
the  TCEA . 


Role  of  Training  Constraints  in  MANPRINT 


Currently,  two  major  sources  of  MANPRINT  r 
tion  exist:  AR  602-2,  MANPRINT,  and  the  draft 
MANPRINT  that  the  revised  TRADOC/AMC  PAM  70-2, 
Acquisition  Handbook  (hereafter  referred  to  as 
TRADOC/AMC  PAM  70-2)  will  include: 


egulatory  i 
chapter  on 
Materiel 
the  Revised 


nforma- 


Training  Constraints.  According  to 
Proof-of-Pr inciple  Phase  will  accomplish 


AR  602-2  (p.  28)  , 
the  following: 


the 


.  .  g.  Special  human  factors  engineering  char¬ 

acteristics,  male  and  female  soldier  character¬ 
istics,  and  manpower,  personnel,  and  training  con¬ 
siderations  peculiar  to  the  system  will  be 
addressed  as  specified  in  the  requirements  docu¬ 
ments  ( AR  71-9).  The  MANPRINT  portion  of  the 
requirements  documents  will  provide  soldier  per¬ 
formance  specif ications  and  consider  maximum  and 
minimum  personnel  aptitudes  and  skill  that  can  be 
required.  .  . 

The  revised  TRADOC/AMC  PAM  70-2  (p.  11.91-11.92),  provides 
more  specific  guidance  on  the  role  of  training  constraints  in 
MANPRINT.  Table  2  lists  this  guidance. 

The  TCEA  will  assist  Army  analysts  in  identifying  one  of  the 
key  constraints  listed  in  Table  2 — namely  time-to-train  limits. 
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Table  1.  Guidelines  for  Describing  TCEA  Products  from 
SSS  DID  (Dl-C MAN-8000 8). 


10.2.5.5.5  Training.  This  subparagraph  shall  be  numbered  3.5.5 
and  specify  the  following  training  requirements: 

a.  Contractor  and  Government  responsibility  for  training 
requirements  that  will  be  generated  by  new  equipment. 
This  subparagraph  shall  also  specify  the  concept  of  how 
training  shall  be  accomplished  (e.g.,  school, 
contractor  training). 

b.  Estimates  of  quantities  of  equipment  being  developed 
that  will  be  required  solely  for  training  purposes. 

c.  The  need  to  develop  associated  training  devices, 
including  types  required.  In  addition,  this  paragraph 
shall  specify:  (1)  detailed  requirements  for  charac¬ 
teristics  of  training  devices,  and  (2)  training  and 
skills  to  be  developed  by  training  devices. 

d .  Training  time  and  locations  available  for  an  effective 
training  program. 

e.  Source  material  and  training  aids  to  support  the 
specified  training. 

f.  Other  training  i equi rements  not  previously  mentioned. 
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Table  2.  Guidance  on  Training  Constraints  from 
Draft  Chapter  11  of  TRADOC/AMC  PAM  70-2. 


EVENT  1:  TRAINING  CONSTRAINTS  AND  EARLY  ANALYSIS 

Descr ipt ion : 

a.  Develop  training  constraints  from  guidance,  lessons  learned, 
Mission  Area  Analysis  (MAA),  post  fielding  Training 
Effectiveness  Analysis  (TEA)  (TRADOC  Reg  350-4),  or 
completed  Early  Comparability  Analysis  (ECA).  Participate 
in  MANPRINT  Joint  Working  Group  (MJWG)  to  ensure  training 
constraints  are  addressed  and  that  sources  of  information 
are  considered.  Continue  to  work  with  the  group  as  the 
acquisition  process  is  executed. 

b.  Accomplish  training  portion  of  ECA  if  not  conducted  as  part 
Of  MAA. 

c.  Provide  a  list  of  constraints  (elimination  of  high  driver 
tasks,  time-to-train  limits,  requirement  for  simulator  to 
meet  constraints  of  training  ammunition,  POL,  etc.)  to 
operational  and  Organizational  (O&O)  Plan,  System  MANPRINT 
Management  Plan  (SMMP)  and  as  a  basis  for  Individual  and 
Collective  Training  Plan  (ICTP)  training  concept 
development. 

d.  Include  additional  constraints  ( communications ,  etc.)  from 
supporting  and  other  participating  schools  and  centers. 

Responsibility:  Directorate  of  Training  and  Doctrine  ( DOTD )  - 

New  System  Training  Office. 


E3-12 


Users 


Overview  of  Users  and  Their  Functions 


Primary  Users .  The  primary  TCEA  user  will  be  the  combat 
developers  within  the  TRADOC  proponent  school  who  produce 
requirements  documents  for  major  systems  (i.e.,  JMSNS,  O&O  Plan, 
LOA,  and  ROC)  and  the  System/Segment  Specification  (SSS)  that 
will  guide  early  contractor  design  activities. 

The  organization  within  the  proponent  school  that  typically 
accomplishes  these  functions  is  the  Directorate  of  Combat 
Development  (DCD). 

Each  DCD  is  organized  slightly  differently.  Portions  of  the 
requirements  documents  and  the  SSS  may  be  completed  by  either  a 
Concepts  and  Studies  Division,  Materiel  Logistics  Support  Divi¬ 
sion,  or  Requirements  Division  within  a  specific  DCD. 

When  we  develop  detailed  specifications  for  the  TCEA,  we 
will  identify  the  specific  DCD  organizations  within  each  major 
TRADOC  proponent  responsible  for  producing  requirements  documents 
and  the  SSS.  This  will  be  accomplished  by  examining  the  AR  10 
series  for  each  school. 

Secondary  Users.  Another  major  user  group  will  be  the  Army 
Material  Command  (AMC)  major  subordinate  command  responsible  for 
providing  the  TRADOC  combat  developer  with  input  to  requirements 
documents.  Since  each  AMC  major  subordinate  command  is  organized 
differently,  the  specific  AMC  organization  that  fulfills  this 
role  can  vary.  Again,  when  we  develop  detailed  design  speci¬ 
fications  for  the  TCEA,  we  will  use  the  AR  10  series  to  generate 
a  detailed  list  of  the  responsible  AMC  organizations.  Typically, 
the  AMC  command  has  an  Advanced  System  Directorate  (ASD)  with  a 
Requirements  Analysis  Division  (RAD).  The  RAD  is  responsible  for 
coordinating  requirements  documents  with  TRADOC.  For  example, 
according  to  AVSCOM  Requirement  10-1  dated  5  March  1986,  the 
Requirements  analysis  Division 

"With  support  from  the  Advanced  Concepts  Division 
( ACD )  and  the  Systems  Technology  Division  (STD)  on 
assigned  projects,  will  maintain  developer  and 
user  interface  with  TRADOC,  FORSCOM,  and  their 
Army  users  and  coordinate  requirements  documents 
such  as  O&Os,  LOAs,  LRs,  and  ROCs  within  AVSCOM." 
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Other  Users.  Other  potential  users  are  the  personnel 
charged  with  reviewing  requirements  documents,  including  HQ 
TRADOC  ( DCSCD ) ,  HQ  AMC  (AMCDRE),  and  the  Requirements  Division 
(DAMO-FOR)  within  DCSOPS;  the  MANPRINT  Policy  Office  within 
ODCSPER  (DAPE-ZAM);  the  MANPRINT  poi nts-of-contact  within  the 
TRADOC  proponent  school  and  AMC  subordinate  command;  and  the  ARI 
field  office  representatives  who  may  provide  MANPRINT  support  to 
TRADOC  schools  or  AMC  subordinate  commands. 


Job  Type 


We  will  develop  the  TCEA  specifically  for  the  primary  users 
(listed  on  the  previous  page)  —  the  combat  developers  within  the 
TRADOC  proponent  school  who  generate  requirements  documents  for 
major  systems  (JMSNS,  O&O  Plan,  LOA,  and  ROC)  and  System  and 
Segment  Specification  (SSS).  The  individuals  who  actually 
perform  these  functions  within  the  assigned  DCD  division  are 
usually  Army  majors  or  captains.  We  will  build  a  more  definitive 
list  of  job  types  when  we  develop  detailed  design  specifications 
for  the  TCEA.  To  accomplish  this,  we  will  contact  the 
appropriate  division  for  DCDs  within  the  major  TRADOC  proponent 
schools . 


Additional  Information  on  Users 


During  the  development  of  the  detailed  design  specifica¬ 
tions,  we  will  gather  additional  information  on  (1)  user  training 
background  and  (2)  current  and  projected  hardware  and  software 
available  to  users.  This  information  will  be  developed  by 
contacting  the  appropriate  division  for  DCDs  for  the  major  TRADOC 
proponent  schools. 


Assumpt ions 

The  following  assumptions  underly  the  development  of  the 

TCEA; 


Major  System  Focus.  The  TCEA  will  describe  training 
constraints  for  major  weapon  systems.  This  means  that  although 
the  general  logic  of  the  TCEA  could  apply  to  other  types  of 
systems,  the  TCEA's  automated  tools  will  only  be  applicable  to 
major  systems. 

Estimate  Constraints  Not  Requirements.  The  TCEA  will  esti¬ 
mate  training  constraints  based  on  the  projections  of  available 
training  resources.  The  TCEA  will  also  estimate  what  the  type 
and  amount  of  training  for  the  new  system  is  likely  to  be. 
However,  the  TCEA  will  not  estimate  what  the  type  and  amount  of 
training  should  be. 


E3-14 


Input  from  other  Products.  To  estimate  training  resources 
for  the  new  system,  the  TCEA  must  receive  input  on  the  system's 
expected  manpower  requirements.  Unless  otherwise  directed,  the 
TCEA  will  automatically  use  the  maximum  manpower  constraints 
produced  by  the  Manpower  Constraints  Estimation  Aid  (MCEA)  as  the 
new  system's  manpower  requirements.  However,  the  TCEA  will  be 
designed  to  allow  users  to  enter  in  other  assumptions  about 
manpower  requirements. 

The  MCEA  will  also  identify  the  "source"  MOSs  for  the  new 
system.  These  source  MOSs  will  describe  the  MOS  populations  from 
which  the  operators  and  maintainers  of  the  new  system  are  likely 
to  be  drawn. 

Product  1,  the  SPREA,  will  provide  an  estimate  of  system 
requirements.  The  estimate  will  describe  the  overall  capability, 
accuracy,  availability,  and  reliability  requirements  for  each 
mission,  and  the  time  and  accuracy  requirements  for  the  mission’s 
individual  operational  functions.  We  assume  that  these  functions 
will  be  broken  out  to  the  level  where  they  are  equivalent  to 
human  operator  task  actions. 

To  estimate  the  number  of  students  who  must  be  trained  in 
institutional  training  courses,  the  TCEA  will  need  estimates  of 
projected  attrition  and  promotion  rates  for  the  new  system's 
source  MOSs.  It  is  assumed  that  these  rates  will  be  provided  by 
Product  3.  However,  procedures  will  also  be  provided  for  using 
current  rates  obtained  from  DMDC  data  files. 

Need  for  Unit  Training  Constraints.  The  TCEA  must  set 
training  time  constraints  for  both  institutional  and  unit  train¬ 
ing.  The  reasons  for  this  are  fairly  obvious.  If  a  maximum 
training  time  constraint  is  set  only  for  institutional  training, 
the  contractor  could  easily  meet  this  constraint  by  assuming  that 
large  numbers  of  tasks  would  be  trained  in  the  unit. 

Differing  Units  of  Analyses  for  Unit  and  Institutional 
Training.  Training  time  constraints  will  be  set  for  each  insti¬ 
tutional  course  likely  to  be  associated  with  the  new  system. 
However,  training  time  constraints  will  be  set  only  for  the 
typical  unit  or  units  providing  unit  training.  This  approach  is 
necessary  because  unit  training  varies  across  units,  and  dealing 
with  constraints  for  all  individual  units  would  not  be  feasible. 

New  Equipment  Training.  The  TCEA  will  not  set  constraints 
for  any  type  of  new  equipment  training. 
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Resources  Available  for  Development .  We  assume  that  there 
are  approximately  nine  person-months  available  for  the  develop¬ 
ment  of  detailed  design  specifications  for  the  TCEA  and 
approximately  48  person-months  available  for  software  coding. 


High-Level  Functional  Requirements 


Technical  Requirements 

Output .  The  TCEA  must  produce  two  distinct  types  of  out¬ 
put.  First,  it  must  produce  estimates  of  the  maximum  amount  of 
time  that  will  be  available  to  train  the  new  system  in  both  ini¬ 
tial  institutional  and  unit  training.  Second,  it  must  generate  a 
set  of  operator  and  maintainer  tasks  for  the  new  system  and 
produce  an  estimate  of  the  likely  amount  of  time  that  will  be 
spent  in  training  these  tasks. 

Role  In  Acquisition  Process.  The  TCEA  information  on 
training  constraints  must  feed  directly  into  Army  requirements 
documents  for  major  systems  (JMSNS,  0&0  Plan,  LOA,  and  ROC)  and 
the  Type  A  speci f ication  that  guides  contractors  designs. 

Users.  The  TCEA  must  be  designed  specifically  for  the 
combat  developers  within  the  TRADOC  proponent  school  who  produce 
requirements  documents  for  major  systems  (JMSNS,  O&O  Plan,  LC\, 
and  ROC)  and  the  Type  A  specification  that  guides  early  contrac¬ 
tor  design  activities. 


Acceptability  and  Usability  Requirements 

The  previous  subsection  presented  an  overview  of  the  tech¬ 
nical  requirements  that  the  TCEA  must  meet.  This  section 
describes  the  acceptability  and  usability  requirements  that  these 
tools  also  must  meet. 

Produce  Tailored  User  Output  and  Processes.  Often  R&D  pro¬ 
ducts  have  not  been  implemented  because  they  failed  to  meet  the 
needs  of  individual  Army  decision  makers.  They  were  R&D  products 
"in  search  of  users."  To  avoid  this  problem  in  the  current 
effort,  we  must  identify  specific  TCEA  users.  Furthermore,  TCEA 
output  should  be  formatted  so  that  Army  users  can  insert  them 
directly  in  MAP  documents  in  order  to  meet  the  requirements  of 
the  new  streamlined  acquisition  process  in  a  timely  fashion. 

Describe  "How  To"  Procedures.  Whenever  possible,  procedures 
should  be  automated  to  reduce  user  analysis  requirements. 

However,  procedures  for  obtaining  input  data  and  '.nterpreting 
results  should  be  presented. 
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Minimize  Organizational  Impacts.  The  TCEA  must  be  designed 
to  fit  the  user  and  not  vice  versa.  Consequently,  the  aid  must 
not  require  additional  personnel  or  cause  restructuring  of  exist¬ 
ing  Army  organizations.  The  TCEA  must  utilize  computer  hardware 
available  at  user  locations  and/or  accessible  via  secure  lines. 
Furthermore,  it  should  use  existing  software  whenever  possible. 

If  it  requires  new  software  packages,  the  cost  of  these  packages 
must  not  exceed  the  users'  typical  software  acquisition  budget. 

Minimize  User  Training.  The  members  of  the  MAP  community 
who  are  expected  to  be  TCEA  users  are  already  overburdened  and 
understaffed.  In  addition,  they  are  trying  to  meet  increasing 
acquisition  requirements  such  as  MANPRINT  within  the  context  of 
the  streamlined  acquisition  process.  Consequently,  training  time 
for  the  (MPT)2  products  must  be  minimized.  This  requires 
development  of  user  interfaces  that  require  no  prior  computer 
experience.  For  example,  the  interface  should  contain  built-in 
job  aids  (e.g.,  help  commands).  Finally,  when  formal  training  is 
required,  it  must  be  developed  in  accordance  with  Army  instruc¬ 
tional  system  design  principles  and  utilize  only  media  that  are 
readily  available  or  accessible  to  users. 

Security.  Since  the  TCEA  may  be  required  to  accept  classi¬ 
fied  data,  it  must  provide  acceptable  levels  of  security. 
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PRODUCT  OVERVIEW 


Output 

The  TCEA  will  produce  two  distinctively  different  types  of 
output.  First  it  will  produce  estimates  of  the  maximum  amount  of 
time  that  will  be  available  to  train  the  new  system.  Second,  it 
will  generate  a  set  of  operator  and  maintainer  tasks  for  the  new 
system  and  produce  estimates  of  the  likely  amount  of  time  that 
will  be  spent  in  training  these  tasks.  The  maximum  training  time 
estimates  will  be  listed  as  constraints  in  the  system  specifica¬ 
tion.  Contractors  will  be  required  to  design  a  system  that  does 
not  require  training  which  exceeds  these  maximum  constraints. 

The  estimates  of  likely  training  time  will  also  be  provided  to 
contractors.  These  estimates  will  be  described  as  guidelines 
that  the  contractors  can  use  to  estimate  the  personnel  character¬ 
istic  requirements  associated  with  their  design.  The  contractors 
will  be  free  to  substitute  other  training  times,  provided  that 
t:.ey  can  provide  a  rationale  for  their  estimate  and  their  total 
training  time  does  not  exceed  the  maximum  training  time 
constraint . 


Integration  with  Other  (MPT)2  Products 

The  first  four  (MPT)2  products,  the  System  Performance 
Requirements  Estimation  Aid  (SPREA),  the  Manpower  Constraints 
Estimation  Aid  (MCEA),  the  Personnel  Constraints  Estimation  Aid 
( PCEA ) ,  and  the  Training  Constraints  Estimation  Aid  (TCEA),  will 
estimate  MPT-related  requirements  and  constraints  during  the 
earliest  phase  of  the  acquisition  process,  the  Requirements  and 
Technology  Base  Activities  Phase. 

The  SPREA  will  help  Army  combat  developers  identify  compre¬ 
hensive,  clear  system  performance  requirements  and  missions.  As 
part  of  the  system  performance  requirements,  the  SPREA  will 
generate  a  list  of  the  operational  functions  that  the  new  system 
must  perform.  The  TCEA  will  use  these  functions  to  generate  an 
operator  task  list  for  the  new  system. 

The  MCEA,  PCEA,  and  TCFA,  will  estimate  manpower,  personnel, 
and  training  constraints,  respectively.  The  manpower  constraints 
the  MCEA  produces  will  describe  the  maximum  crew  sizes  and  the 
maximum  total  number  of  people  who  will  be  available  to  man  the 
new  system.  The  TCEA  will  use  the  latter  to  determine  how  many 
people  must  be  trained  on  the  new  system.  The  MCEA  will  also 
identify  "source"  MOSs  for  the  new  system.  These  are  the  MOSs 
from  which  the  new  system's  operators  and  maintainers  will 
probably  be  drawn. 
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The  PCEA  will  identify  the  maximum  level  of  personnel 
characteristics  that  can  be  supported,  given  the  expected  future 
distribution  of  these  characteristics  in  the  Army.  The  PCEA  will 
also  identify  expected  attrition  and  promotion  rates.  The  TCEA 
will  also  use  these  rates  to  estimate  the  number  of  people  who 
must  be  trained  on  the  new  system. 

The  last  two  (MPT)2  products,  the  Manpower  Determination  Aid 
( MDA )  and  the  Personnel  Requirements  Estimation  Aid  (PREA),  will 
assist  in  the  evaluation  of  contractor  designs  during  the  Proof- 
of-Principle  Phase.  The  MDA  will  estimate  the  manpower  require¬ 
ments  associated  with  contractor  designs.  The  PREA  will  estimate 
the  personnel  characteristics  required  to  support  these  designs. 
The  likely  training  time  estimates  per  task  that  the  TCEA 
produces  will  feed  both  these  products. 


Two  Types  of  TCEA  Output 

The  TCEA  will  produce  two  related  but  distinctively 
different  outputs:  (1)  estimates  of  maximum  training  time  for 
initial  institutional  and  unit  training  and  (2)  estimates  of  the 
likely  amount  of  training  time  that  will  be  spent  in  training 
each  new  system  task. 

Our  approach  for  generating  these  two  types  of  estimates  is 
quite  different.  To  estimate  maximum  training  time,  we  propose 
to  assess  the  resources  that  will  be  available  to  train  the  new 
system.  To  estimate  likely  training  time,  we  propose  to  develop 
a  prediction  model  that  captures  the  current  training  time 
decision  process  in  each  mission  area.  The  predictors  in  this 
model  will  be  the  training  factors  currently  used  to  determine 
what  tasks  should  be  trained  in  institutional  and  unit  training 
(for  example,  training  difficulty,  consequences  of  inadequate 
performance ) . 


Our  Approach  for  Estimating  Maximum  Training  Time 
Our  General  Approach  to  Constraints 

The  MA&D/DRC  team  uses  the  same  concept  of  "constraint"  in 
each  of  the  three  constraint-related  MPT  products  (Products  2,  3, 
and  4).  Our  basic  approach  in  tnese  products  is  to  estimate 
available  resources  and  then  to  use  these  resources  to  constrain 
the  new  system  design.  Available  resources  are  identified  by 
assessing  (1)  the  resources  currently  associated  with  the  systems 
that  the  new  system  will  replace  and  (2)  policy  or  other  related 
changes  that  will  change  what  resources  will  be  available  in  the 
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future.  Each  of  the  three  constraint  products  estimates  a 
different  type  of  available  resource.  The  MCEA  (Product  2)  esti¬ 
mates  available  manpower  slots;  tne  PCEA  (Product  3)  estimates 
the  type  of  people  who  will  be  available;  and  the  TCEA  (Product 
4)  estimates  maximum  training  time  that  can  be  supported,  given 
available  training  resources. 

Our  approach  to  MPT  constraints  is  directly  congruent  with 
the  Army's  concept  of  "zero  sum"  resourcing.  This  concept  forces 
new  system  developers  to  describe  specifically  where  (i.e.,  which 
systems  or  units)  the  resources  for  their  new  system  will  come 
from.  A  new  system  may  not  require  more  resources  than  the 
system  it  replaces.  However,  if  it  does,  the  proponent  must 
describe  the  source  of  these  resources. 


How  the  Army  Sets  Training  Constraints 

Before  discussing  our  approach  to  setting  training  con¬ 
straints  for  a  particular  system,  we  will  review  the  Army’s 
general  processes  for  setting  institutional  and  unit  training 
constraints . 

Institutional  Training  Constraints.  Institutional  training 
requirements  for  a  given  year  are  reviewed  during  the  Structured 
and  Manning  Decision  Review  (SMDR).  The  SMDR  is  part  of  the 
Planning,  Programming,  Budgeting,  and  Execution  System  (PPBES). 
Table  3  describes  the  participants  in  the  SMDR.  During  the  SMDR, 
manning  and  associated  institutional  training  requirements  are 
verified.  The  resulting  training  requirements  for  each  course 
are  then  compared  with  the  available  training  resources.  Courses 
lacking  sufficient  resources  to  train  their  full  training 
requirements  are  termed  "constrained."  These  constraints  are 
resolved  by  taking  resources  from  other  courses  or  lowering  the 
course  training  requirements  (lowering  the  number  of  people  who 
must  be  trained).  The  result  is  a  recommended  training  program. 
The  final  training  program  for  a  given  fiscal  year  is  published 
in  the  Army  Program  for  Individual  Training  (ARPRINT). 

The  Army  has  developed  an  automated  tool  called  the  Army 
Training  Requirements  and  Resources  System  (ATRRS)  to  assist  in 
identifying  and  tracking  training  requirements  and  constraints 
(see  AR  350-10) . 

ATRRS  allows  the  Army  to  match  and  compare  training  require¬ 
ments  with  training  resources. 

ATRRS  draws  information  from  and  provides  information  to 
four  levels  of  command: 
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Table  3.  List  of  Participants  In  SMDR. 


(1 )  Office  of  the  Deputy  Chief  of  Staff  for  Military  Operations  and  Plans  (ODCSOPS) 

(2)  Office  of  the  Deputy  Chief  of  Staff  for  Personnel  Operations  and  Plans  (ODCSPER) 

(3)  Office  of  the  Deputy  Chief  of  Staff  for  Logistics  (ODSCIOG) 

(4)  National  Guard  Bureau  (NGB) 

(5)  Office  of  the  Chief,  Army  Reserve  (OCAR) 

(6)  United  States  Army  Military  Personnel  Center  (MILPERCEN) 

(7)  United  States  Army  Training  and  Doctrine  Command  (TRADOC) 

(8)  United  States  Army  Recruiting  Command  (USAREC) 

(9)  United  States  Army  Health  Services  Command  (HSC) 

(10)  Installations  where  the  specific  courses  are  taught  (proponent  schools) 
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Department  of  the  Army  Staff 


•  Military  Personnel  Center  and  its  Reserve  Component 
counterparts 

•  The  Army’s  school  systems 

•  The  Army's  schools  and  training  centers 

The  ATRRS  data  base  maintains  information  at  the  course 
level  of  detail  on  all  courses  taught  by  or  for  Army  personnel. 

It  produces  reports,  analyses,  graphs,  and  selected  data  displays 
on  requirements,  entrants,  graduates,  training  loads,  and  other 
related  information. 

One  of  ATRRS 's  major  products  is  the  ARPRINT.  The  ARPRINT 
is  a  major  resourcing  document  for  the  U.S.  Army  training  and 
Doctrine  Command  (TRADOC),  the  Surgeon  General  (TSG),  DoD 
Schools,  and  other  agencies.  The  ATRRS  is  also  the  basis  for 
developing  class  schedules. 

The  ATRRS  System  provides  accurate,  timely,  and  responsive 
training  input  data  for  the  PPBES.  ATRRS  supports  the  PPBES  by 
providing  information  for  the  Army  Staff  to  use  in  responding  to 
OSD,  OMB,  and  Congressional  inquiries.  A  key  document  the  ATRRS 
produces  in  support  of  the  PPBES  is  the  Military  Manpower  Train¬ 
ing  Report  ( MMTR ) .  The  DoD  submits  the  MMTR  to  Congress  to 
support  the  Army's  request  for  authorization  of  student  training 
loads  in  individual  training. 

ATRRS  calculates  projected  training  input  and  loads  that  are 
used  as  a  basis  for  providing  resources  for  the  Army’s  school 
system.  In  addition,  ATRRS  serves  as  a  mechanism  for  HQDA  to 
correlate  training  requirements  to  the  Army's  recruiting 
objectives . 

ATRRS  has  interactive  terminals  at  training  MACOMS,  agen¬ 
cies,  schools,  and  training  centers  throughout  the  Army.  Accord¬ 
ing  to  the  ATRRS  user's  manual,  ATRRS  "can  be  run  on  any  computer 
system  that  supports  Customer  Information  Control  System  (CICS), 
Virtual  System  Access  Method  (VSAM),  and  telecommunications. 

The  ATRRS  data  base  contains  several  data  elements  that  are 
particularly  relevant  to  the  TCEA,  including  descriptions  of 
institutional  course  lengths  and  the  course  annual  trainee  input 
capacity.  The  latter  describes  the  maximum  number  of  students 
that  can  be  trained  in  a  course,  given  available  resources  (e.g., 
classroom  space,  training  facilities). 
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Unit  Training  Constraints.  Each  unit  must  pay  for  its  own 
unit  training  by  allocating  resources  from  the  Command  Operating 
Budget  (COB),  which  commanders  use  to  prepare  long-range  plans 
for  unit  training.  (For  more  information  on  the  COB,  see  FM  25- 
2,  Unit  Training  Management.)  The  COB  is  based  on  input  from 
battalions  and  separate  companies.  The  input  is  consolidated  at 
higher  levels  between  March  and  June  of  each  year  and  forwarded 
to  the  Army  (DA)  during  July.  The  Army  has  developed  an  auto¬ 
mated  data  base  called  the  Training  Management  Control  System 
(TMACS)  to  assist  unit  commanders  in  estimating  resource  require¬ 
ments  for  unit  training  events.  Appendix  C  provides  a  more 
detailed  description  of  TMACS  and  the  unit  training  resource 
planning  process. 

Many  factors  limit  the  time  spent  on  unit  training.  An  ARI 
study  by  Johnson,  et  al.  (1982)  systematically  identified  the 
factors  detracting  from  unit  training  in  a  FORSCOM  infantry 
division.  Table  4  provides  a  rank-ordered  listing  of  these 
detractors.  Within  the  limited  time  available  for  unit  training, 
many  requirements  must  be  trained.  System-specific  and  individu¬ 
al  sustainment  training  are  only  a  subset  of  these  requirements. 
For  example,  each  battalion  receives  a  long  list  of  directed 
training  requirements  from  each  of  the  command  levels  in  which  it 
is  embedded.  Another  ARI  study  conducted  in  1979  by  Buxton, 
Miller,  and  Hayes  determined  the  amount  of  time  available  per 
week  for  individual  sustainment  training  in  the  unit.  The 
researchers  concluded  that  with  "intensive  management"  only  about 
two  days  per  week  or  100  days  per  year  could  be  allotted  to  unit 
sustainment  training.  However,  they  calculated  that  180  days  per 
year  were  needed  to  meet  the  unit  sustainment  training 
requirements. 

Unfortunately,  currently  no  automated  data  base  exists  to 
describe  unit  training  resource  constraints  or  the  actual  time 
spent  on  unit  training  events — the  key  unit  training  data 
elements  needed  by  the  TCEA. 

The  Army's  automated  tool  for  unit  training  resource  plan¬ 
ning,  TMACS,  has  several  features  that  limit  its  utility  for  the 
TCEA.  First,  TMACS  does  not  distinguish  system-specific  training 
events  from  other  training  events.  Second,  TMACS  does  not  record 
information  on  the  actual  time  spent  on  training  events.  Third, 
TMACS  does  not  contain  information  on  training  resource 
constraints . 

The  Army  Development  and  Employment  Agency  (ADEA)  is 
developing  an  automated  information  system  called  the  Integrated 
Training  Management  System  (ITEMS).  ITEMS  is  designed  to  support 
the  integration  of  seven  unit  training  management  functional 
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Table  4.  Rank  Ordering  of  Training  Detractors 
within  the  71D.* 


Problem  Areas 

Rank 

Shortage  of  NCOS  and  Critical  MOS« 

1 

Turbulence 

2 

Personnel  Shortages 

3 

Characteristics  of  Incoming  Personnel 

4 

Time  Spent  In  Personnel  Management 

5 

Tasklngs 

6 

Training  Management 

7 

Information  Management 

8 

Administrative  Tasks 

9 

Maintenance 

10 

Supply 

11 

Inspections 

12 

Inprocesslng/Outprocesslng 

13 

Redundancy 

14 

Schools 

15 

•Johnson,  C.,  Funk,  S.,  Elliot,  M.,  Msilzi,  L,  &  Hlllar,  J.  (1982). 
Development  and  Implementation  management  Innovations 
(ARI  Research  Report  1357). 
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areas:  (1)  Requirements,  (2)  Calendars  and  Schedule,  (3)  Training 
Tempo,  (4)  Funds,  (5)  Ammunition,  (6)  Training  Areas,  Ranges,  and 
Facilities,  and  (7)  Training  Aids  and  Devices.  The  ITEMS  will 
contain  information  on  the  actual  time  spent  per  unit  training 
event  and  should  contain  enough  information  to  identify  which  of 
these  events  deal  with  system-specific  training  (ITEMS  Data  Base 
Specification,  May  1986).  The  ITEMS  will  not  describe  resource 
constraints;  however  it  will  describe  all  required  unit  training 
tasks,  including  tasks  directed  by  higher  commands.  The  major 
limitation  of  the  ITEMS  system  is  that  it  is  still  in  develop¬ 
ment.  It  is  scheduled  for  implementation  in  the  FY  88-90 
timeframe  but  has  undergone  a  number  of  schedule  slippages. 

Because  of  the  uncertain  ITEMS  development  schedule  we  have 
decided  not  to  include  the  ITEMS  as  a  TCEA  data  source. 

Summary .  In  both  institutional  and  unit  training,  the  Army 
determines  "training  constraints"  by  looking  at  available 
resources  (for  example,  cost,  time,  classroom  space).  As  the 
sections  that  follow  will  demonstrate,  our  concept  for  setting 
maximum  training  time  limits  is  directly  congruent  with  this 
approach.  There  may  be  other  approaches  for  determining  maximum 
training  time  during  the  early  phases  of  the  acquisition  process, 
but  only  a  resource-based  approach  is  likely  to  have  meaning  to 
the  Army  personnel  who  must  actually  identify  training 
constraints  early  in  the  acquisition  process. 


How  We  Will  Determine  Maximum  Initial  Insti tutional  Training  Time 

Our  concept  for  determining  maximum  initial  institutional 
training  time  is  as  follows.  First,  the  TCEA  will  extract 
initial  institutional  training  course  information  from  ATRRS  for 
all  the  source  MOSs  associated  with  the  new  system.  These  source 
MOSs,  which  will  be  identified  by  Product  2,  will  describe  the 
MOSs  from  which  the  operators  and  maintainers  of  the  new  system 
will  be  drawn.  The  TCEA  will  then  calculate  the  maximum  number 
of  training  man-days  that  can  be  supported  in  these  courses, 
given  available  resources,  using  the  following  algorithm: 

Maximum  Current  Course  Trainee 

Available  =  Course  X  Input  Capacity 

Training  Day  Length 

The  course  trainee  input  capacity  describes  the  maximum 
number  of  students  that  can  currently  be  trained  in  a  course, 
given  available  training  resources. 

Next  the  TCEA  will  determine  the  maximum  length  for  the  new 
system  course  using  the  following  algorithm: 
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Maximum  Available  Required  Annual 

Training  =  Training  '  Student 

Time  Days  Load 

The  required  student  load  is  the  annual  number  of  students 
who  must  be  trained  in  each  institutional  training  course  to 
sustain  the  new  system's  projected  manpower  requirements.  These 
estimates  are  determined  by  calculating  how  many  people  in  the 
new  system's  projected  manpower  slots  can  be  expected  to  leave 
those  slots  each  year  based  on  projected  attrition  and  promotion 
rates.  To  estimate  the  new  system's  manpower  requirements,  the 
TCEA  will  use  the  maximum  manpower  constraints  produced  in 
Product  3.  The  result  will  be  a  worst-case  estimate  of  student 
load  (the  user  will  also  be  given  the  option  of  inputting  his  or 
her  own  manpower  requirement  estimates).  We  assume  that 
projected  attrition  and  promotion  rates  will  be  produced  by 
Product  2.  If  projected  rates  are  not  available,  data  on 
existing  rates  from  DMDC  or  FORECAST  can  be  used. 

For  many  MOSs,  particularly  many  maintenance  MOSs,  the 
institutional  training  course  provides  training  on  several 
different  systems.  During  the  application  of  TCEA,  the  user  will 
examine  the  initial  training  Programs  of  Instruction  (POIs)  and 
identify  the  time  currently  spent  on  training  weapon  systems  that 
will  still  be  operational  when  the  new  system  is  fielded.  This 
time  is  assumed  to  be  a  fixed  constraint  that  the  new  system 
cannot  change.  The  time  will  be  subtracted  from  the  estimated 
initial  training  course  length  to  estimate  the  time  that  the 
initial  training  course  will  spend  in  training  the  new  system. 


How  We  Will  Determine  Maximum  Unit  Training  Time 

To  estimate  maximum  training  time  per  unit  type,  we  will 
develop  a  data  base  called  the  Unit  Training  Time  Constraint  Data 
Base.  This  data  base  will  list  the  maximum  amount  of  time  that 
different  unit  types  are  expected  to  have  for  weapon  system- 
specific  training.  The  estimates  will  be  broken  out  by  mission 
area  and  by  unit  location.  During  TCEA  application,  the  system 
will  link  the  unit  types  that  will  get  the  new  system  to  the  unit 
types  in  the  Unit  Training  Time  Constraint  Data  Base.  This 
linkage  will  produce  an  estimate  of  maximum  unit  training  time. 

It  is  assumed  that  Product  2  will  provide  a  description  of  the 
unit  types  that  will  get  the  new  system.  If  Product  2  does  not 
provide  this  information,  it  should  be  readily  available  in  the 
draft  O&O  Plan.  In  many  cases,  a  particular  unit  type  can  be 
expected  to  operate  or  maintain  more  than  one  weapon  system.  In 
these  cases,  the  TCEA  will  simply  divide  the  maximum  training 
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time  estimate  by  the  number  of  weapon  systems  expected  to  be 
operated  or  maintained  by  the  unit  types  getting  the  new  system. 

Unfortunately,  the  information  needed  for  the  Unit  Training 
Time  Constraint  Data  Base  is  not  available  in  an  automated  data 
base.  To  develop  this  information,  we  propose  to  send  out  a 
survey  questionnaire  to  a  representative  sample  of  unit  training 
managers.  This  survey  will  be  designed  to  obtain  all  of  the  unit 
training  time  information  for  the  TCEA's  data  bases  and  models. 
Table  5  lists  the  types  of  questions  the  survey  will  include. 

The  survey  will  be  developed  in  accordance  with  state-of- 
the-art  guidelines  for  mail  surveys  (Altschuld  &  Lower,  1984; 
Dillman,  1978;  Dyer,  et  al.,  1976;  Eudos,  1983;  Sudman  & 

Bradburn,  1982;  Lockhart,  1984).  DRC  has  successfully  conducted 
a  number  of  surveys  for  ARI.  These  surveys  include  studies  on 
the  Joint  Optical  Information  Network  (JOIN)  (Lockhart,  1987), 
Analysis  of  the  Soldier  in  Europe  Survey  Data  (Lockhart,  1987), 
and  the  user  survey  for  the  Man  Integrated  System  Technology 
(MIST)  (Dynamics  Research  Corporation,  1985). 


Our  Approach  for  Estimating  Likely  Training  Time  per  Task 

To  estimate  likely  training  time,  we  propose  to  develop  a 
prediction  model  that  captures  the  current  training  time  decision 
process  in  each  mission  area.  The  predictors  in  this  model  will 
be  the  training  factors  currently  used  to  determine  what  tasks 
should  be  trained  in  institutional  and  unit  training.  Before  we 
can  estimate  likely  training  time  per  new  system  task  however,  we 
must  develop  a  list  of  operator  and  maintainer  tasks.  Before  we 
can  generate  this  task  list,  we  must  identify  the  new  system 
hardware  and  software.  In  the  sections  that  follow,  we  describe 
our  procedures  for  developing  a  generic  hardware  and  software 
list  for  the  new  system  and  for  generating  a  preliminary  set  of 
operator  and  maintainer  tasks.  We  follow  that  with  a  description 
of  the  training  time  prediction  model. 


We  have  specifically  attempted  to  develop  an  approach  for 
estimating  likely  training  time  that  minimizes  the  number  of  data 
items  that  must  be  created  by  the  user.  It  is  important  to  point 
out  that  the  TCEA  will  be  applied  during  the  Requirements/ 
Technology  Base  Activity  Phase  of  the  acquisition  process.  At 
this  point  in  the  acquisition  process,  the  user  will  not  have 
detailed  information  on  the  contractor  design.  Developing 
extensive  amounts  of  training  data  for  each  task  at  this  point  in 
the  acquisition  process  would  not  be  efficient  because  it  is 
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Table  5.  Type  of  Questions  to  Be  Included  In  Unit  Training  Time  Survey. 
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•These  are  not  the  actual  questions  that  will  be  Included  In  the  survey.  They  are  the  topics  that  the  questions  win  cover. 


likely  that  large  amounts  of  these  data  would  have  to  be  revised 
to  reflect  the  designs  eventually  produced  by  contractors.  At 
this  point  in  the  acquisition  process,  we  want  to  provide  guid¬ 
ance  to  the  contractor — we  do  not  want  to  develop  the  new 
system's  training  program. 


Development  of  A  Generic  Hardware  and  Software  List 

•The  TCEA  will  help  the  user  generate  a  generic  hardware  and 
software  list  for  the  new  system.  The  TCEA  will  present  the  user 
with  a  list  of  the  major  equipment  items  associated  with  the 
systems  to  be  replaced  by  the  new  system.  (Product  2  will 
identify  these  replacement  systems.)  The  user  will  add  or  delete 
items  from  this  list  to  reflect  the  new  system's  mission,  func¬ 
tions,  and  performance  requirements  and  replace  references  to 
specific  equipment  items  with  references  to  generic  equipment 
(e.g.,  replace  reference  to  T999  engine  with  reference  to  diesel 
engine) . 


Development  of  Maintenance  Task  List 

We  assume  that  the  three  other  (MPT)^  products  that  will  be 
applied  during  the  Requirements/Technology  Base  Activities  Phase 
will  not  produce  a  list  of  maintenance  tasks  for  the  new  system. 
Product  1,  the  SPREA,  will  provide  information  on  overall  system 
performance  requirements  for  both  operations  and  maintenance. 

The  SPREA  will  estimate  the  low-level  operational  functions 
associated  with  a  system's  mission.  It  is  expected  that  these 
functions  can  be  mapped  onto  operator  task  actions  on  a  one-to- 
one  basis.  However,  we  believe  that  Product  1  will  not  be  able 
to  estimate  low-level  maintenance  functions — functions  at  the 
maintenance  task  level.  It  is  impossible  to  identify  maintenance 
tasks  until  operational  functions  have  been  allocated  to 
particular  types  of  equipment.  The  performance  requirements  for 
a  maintenance  task,  and  indeed  even  the  maintenance  tasks  them¬ 
selves,  will  depend  on  what  must  be  maintained.  One  of  the 
requirements  of  Product  1  is  that  it  must  estimate  functional 
performance  requirements  without  any  function  allocation.  Thus, 
in  this  context,  it  is  unlikely  that  Product  1  will  estimate 
maintenance  tasks. 

To  produce  a  list  of  maintenance  tasks,  the  TCEA  will  access 
the  AMMDB  to  obtain  a  list  of  the  maintenance  tasks  associated 
with  the  system(s)  that  the  new  system  will  replace.  A  prelimi¬ 
nary  maintenance  task  list  for  the  new  system  will  then  be  gener¬ 
ated  by  linking  the  maintenance  actions  from  the  existing  equip¬ 
ment  with  the  new  system  generic  equipment  items.  The  user  will 
then  have  an  opportunity  to  review  and  update  this  list. 
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Development  of  Operator  Task  List 


We  assume  that  Product  1  will  produce  a  list  cf  operational 
functions.  A  subset  of  these  functions  is  expected  to  be  the 
same  as  operator  task  actions.  The  TCEA  will  present  the  user 
with  an  operational  task  action  and  a  menu  displaying  the  generic 
equipment  items.  The  user  will  be  asked  to  indicate  which  items 
on  this  menu  will  be  used  in  performing  the  operational  task. 


The  Training  Time  Prediction  Model 


To  estimate  training  time  per  task,  we  propose  to  develop  a 
prediction  model  that  captures  the  current  training  time  decision 
process  in  each  mission  area.  The  model  will  contain  a  set  of 
regression  equations  that  predict  training  time  per  task  as  a 
function  of  a  set  of  "training  factors."  The  tr: ining  factors 
will  attempt  to  represent  the  variables  actually  used  by  training 
developers  to  determine  the  time  to  train  each  task  in  institu¬ 
tional  and  unit  training.  With  this  in  mind,  we  propose  to  use 
the  same  nine  training  factors  the  Army  currently  uses  to  deter¬ 
mine  what  tasks  should  be  trained  in  institutional  and  unit 
training.  These  factors  are: 


1)  Percent  of  members  performing 

2)  Average  percent  of  time  spent  by  members  performing 

3)  Task  learning  difficulty 

4)  Consequences  of  inadequate  performance 

5)  Task  delay  tolerance 

6)  Probability  of  deficient  performance 

7)  Immediacy  of  performance 

8)  Relative  frequency 

9)  Training  emphasis 


These  nine  factors  are  often  combined  to  estimate  the 
training  "criticality"  of  each  task.  Ratings  on  these  training 
factors  are  collected  as  part  of  The  Army  Occupational  Survey 
Program  (AOSP).  The  AOSP  maintains  a  large  data  base  of  these 
factor  ratings  for  individual  tasks. 


Many  different  training 
ing  time  prediction  model, 
better  factors  than  the  ones 
portion  of  the  TCEA  we  are  a 
ing  time  for  the  new  system 
should  be.  Thus,  the  TCEA  u 
Army  training  analysts  are  1 


factors  could  be  used  in  the  train- 
Some  would  argue  that  there  are  even 
the  AOSP  uses.  However,  in  this 
ttempting  to  estimate  what  the  train- 
tasks  i^s  likely  to  be,  not  what  it 
ses  the  training  factors  that  the 
ikely  to  use. 


E3-31 


Description  of  AOSF  Training  Factors.  The  Army  Occupational 
Survey  Program  (AOSP)  has  routinely  collected  training  factor 
data  for  enlisted  MOSs  since  1981.  The  primary  purpose  of 
collecting  the  training  factor  data  is  to  help  training  course 
developers  decide  which  tasks  should  be  trained  (either  in  insti¬ 
tutional  training  or  in  supervised  on-the-job  unit  training). 
Eight  of  the  nine  factors  used  in  this  decision  process  are 
derived  from  the  Instructional  System  Development  eight-factor 
model  for  making  training  decisions.  The  ninth  factor,  training 
emphasis,  is  derived  from  recent  Air  Force  research  on  training 
task  selection.  (This  research  is  discussed  below.) 

Training  factor  data  are  stored  in  the  automated  Compre¬ 
hensive  Occupational  Data  Analysis  Programs  (CODAP)  data  bases. 

A  recent  study  by  Goldman  (1985)  assessed  the  reliability  of 
the  nine  training  factor  scales  and  examined  their  success  in 
predicting  what  tasks  were  actually  trained  (which  AOSP  tasks 
were  actually  designated  critical  and  included  in  the  Soldier's 
Manual).  Goldman  found  that  the  training  factors  were  highly 
intercor related  and  that  only  two  factors,  training  emphasis  and 
consequences  of  inadequate  performance,  were  needed  to  predict 
which  tasks  were  critical.  These  results  parallel  results  from  a 
previous  Air  Force  occupational  survey  study  by  Ruck,  Thompson, 
Brown,  &  Stacey  (1978). 

The  results  of  Goldman  and  Ruck,  et  al.  are  encouraging 
because  they  suggest  that  only  a  few  of  the  training  factors  may 
be  needed  to  predict  training  time  (at  least  for  institutional 
training).  Using  fewer  factors  is  desirable  because  it  would 
reduce  the  size  of  the  TCEA  data  bases  and  user  input 
r equi r ements . 

Development  of  the  Training  Time  Prediction  Models.  To  de¬ 
velop  the  training  time  prediction  equations,  a  series  of  re¬ 
gression  analyses  will  be  conducted  during  TCEA  development  using 
the  training  factors  as  predictors  and  the  time  to  train  a  task 
as  a  criterion.  Separate  equations  will  be  developed  to  predict 
institutional  and  unit  training  time  within  each  mission  area. 

To  develop  the  data  for  the  regression  analyses,  we  will 
select  a  sample  of  tasks  for  each  mission  area  from  the  CODAP 
data  base.  Data  on  the  nine  training  factors  will  be  extracted 
from  this  data  base.  We  will  also  obtain  data  on  the  time  spent 
training  the  tasks  in  initial  training  by  examining  the  POI  for 
the  initial  institutional  training  course.  (The  Trainer's  Guide 
for  the  MOS  associated  with  the  task  identifies  this  course.) 
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Estimates  of  the  annual  amount  of  time  spent  in  unit 
training  will  be  obtained  from  a  survey  questionnaire  sent  to 
unit  training  managers.  This  survey  will  also  be  used  to 
determine  the  maximum  time  available  to  train  all  system  tasks. 

A  more  detailed  description  of  the  procedures  that  will  be 
employed  in  this  survey  appears  in  the  section  entitled  MA&D/DRC 
Approach  for  Development,  p.  51. 

Once  data  on  training  time  have  been  collected,  a  series  of 
step-wise  regression  analyses  will  be  conducted  using  the  train¬ 
ing  factor  variables  as  predictors  and  the  unit  and  institutional 
training  time  measures  as  criterion  variables.  In  conducting  the 
regression  analyses,  we  will  attempt  to  identify  the  minimum 
number  of  training  factor  variables  needed  to  predict  training 
time  for  each  mission  area.  By  minimizing  the  number  of  training 
factor  variables,  we  can  reduce  the  number  of  ratings  which  must 
be  provided  by  users. 

Application  of  the  Training  Time  Prediction  Equations.  The 
TCEA  will  assist  the  user  in  inputting  data  on  the  training 
factors  which  are  included  in  the  training  prediction  model. 

Once  these  data  are  entered,  the  TCEA  will  apply  the  regression 
equations  to  predict  institutional  and  unit  training  time. 

To  assist  the  user  in  developing  the  training  factor  scores, 
the  TCEA  will  help  the  user  identify  ’’similar''  tasks  which 
already  have  training  factor  ratings.  The  TCEA  will  extract 
these  ratings  from  CODAP  and  store  them  in  an  on-line  data  base 
called  the  Training  Factor  Rating  Data  Base.  The  system  will 
automatically  identify  a  "similar”  task  based  on  similarities  of 
task  action  and  noun  statements  to  categories  in  Task  and  Equip¬ 
ment  Taxonomies.  The  user  can  review  or  modify  tnis  assignment 
or  select  another  task  from  the  Training  Factor  Rating  Data 
Base.  Finally,  the  user  will  be  able  to  review  the  training 
factor  ratings  associated  with  the  similar  tasks  and  update  them 
to  reflect  the  new  system's  equipment  design  and  system  per¬ 
formance  requirements. 


Breaking  Out  Likely  Training  Time  Estimates  among  Training  Types 

In  addition  to  estimating  the  total  amount  of  time  likely  to 
be  spent  training  a  task,  the  TCEA  will  also  determine  the  amount 
of  time  spent  training  a  task  using  different  "types"  of  train¬ 
ing.  Different  categories  of  training  types  will  be  used  to 
characterize  initial  institutional  and  unit  training. 

In  initial  institutional  training,  "training  type"  will 
refer  to  different  methods  and  media.  To  describe  training 
methods  we  will  use  the  taxonomy  of  training  methods  TRADOC  uses 
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in  its  POls.  We  will  use  the  taxonomy  displayed  in  Table  6  to 
describe  training  media.  This  taxonomy  primarily  distinguishes 
between  different  types  of  training  devices.  We  propose  this 
approach  because  training  devices  are  the  most  costly  media  and 
have  had  the  biggest  demonstrated  impact  on  training  task 
performance . 

In  unit  training,  "training  types"  will  refer  to  (a) 
training  level  --  either  individual  sustainment  training  or 
collective  ♦’gaining,  and  (b)  type  of  training  media.  The  same 
media  categories  used  for  institutional  training  will  be  used  for 
unit  training. 


Our  Approach  for  Resolving  Differences  Between  Maximum 
Available  Training  Time  and  Estimated  Likely  Training  Time 

The  TCEA  will  aggregate  the  likely  training  time  estimates 
for  each  task  to  produce  estimates  of  total  likely  training  time 
for  each  MOS.  Separate  estimates  will  be  developed  for 
institutional  and  unit  training.  These  times  will  be  compared  to 
the  maximum  time  constraints.  A  report  will  be  generated  listing 
the  MOSs  that  have  shortfalls  (have  estimated  training  times  that 
exceed  the  maximum  time  available).  The  report  will  also 
describe  the  size  of  the  shortfall  and  the  associated  maximum  and 
likely  training  time  estimates. 

The  TCEA  will  then  present  the  user  with  a  prioritized  list 
of  the  input  parameters  that  can  be  changed  to  eliminate  a 
training  time  shortfall.  The  user  will  select  the  parameters  to 
be  modified.  The  TCEA  will  then  guide  the  user  through  the  steps 
needed  to  enter  new  values  for  these  parameters  and  calculate 
their  impact  of  training  time  shortfalls. 


Steps 

Applying  the  TCEA  involves  14  steps.  The  first  six  steps 
estimate  maximum  training  time  for  initial  institutional  and  unit 
training.  The  next  six  steps  estimate  the  likely  training  time 
for  each  generic  new  system  task.  In  the  last  two  steps,  the 
likely  training  times  are  combined  and  compared  with  the  maximum 
training  time  constraints,  shortfalls  are  identified,  and  TCEA 
steps  are  iterated  until  the  shortfalls  are  eliminated. 


Steps  Involved  in  Estimating  Maximum  Training  Time 

Figure  2  provides  an  overview  of  the  six  steps  the  user  will 
go  through  to  generate  maximum  training  time  estimates. 
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Table  6.  Example  of  Media  Types 


Armor 

Conduct-of-Fire  Trainers 

Tank  Trainers  (Driving/Gunnery/Turret) 

Tracked  Vehicle  Driving  Tminer 

Weapons  &  Ordnance 

Firing  Simulator  Systems 

Signal 

Countermeasures  Training  Signal  Transmitting  Set 
Digital  Computer  Trainer 

Aviation 

Flight  Simulator 
Composite  Trainer 
Armament  Systems  Trainer 
Hydraulic  &  Electrical  Systems  Trainer 
Flight  Control  System  Trainer 
Panel  Trainers 

Instrument  &  Display  Trainers 
Radar  Operator  &  Target  Simulator 
Power  Plant/Drive  System  Trainers 
Ejection  Seat  Trainer 
Cockpit  Procedures  Trainers 

Air  Defense  Artillery 

Armament  Maintenance  Trainer 
Guided  Missile  Intercept  *  Aerial  Trainer 
Guided  Missile  System  Radar  Signal  Simulator 

Field  Artillery 

Field  Artillery  Trainer 
Programmer  •  Test  Station 
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Figure  1 .  Steps  Involved  in  Estimating  Maximum  Training  Time. 


During  the  first  of  these  steps,  the  TCEA  will  extract 
institutional  training  course  data  from  the  Army  Training 
Requirements  and  Resources  System  ( ATRRS ) .  These  data  will 
include  (a)  a  list  of  the  initial  institutional  training  courses 
associated  with  the  MOSs  from  which  the  operators  and  maintainers 
of  the  new  system  will  be  drawn,  (b)  the  length  of  these  courses, 
and  (c)  the  course  trainee  input  capacity. 

During  the  second  step,  the  user  will  examine  the  initial 
training  POIs  and  identify  the  time  that  is  currently  spent  on 
training  weapon  systems  that  are  expected  to  be  operational  at 
the  same  time  as  the  new  system.  This  time  is  assumed  to  be  a 
fixed  constraint  that  cannot  be  changed  by  the  new  system.  (In 
Step  5,  this  time  will  be  subtracted  from  the  estimated  initial 
training  course  length  to  produce  an  estimate  of  the  time  that 
will  be  spent  training  the  new  system  in  the  initial  training 
course . ) 

During  the  third  step,  the  system  will  estimate  the  maximum 
available  training  man-days  for  each  institutional  training 
course  by  multiplying  the  course  length  by  the  training  input 
capacity . 


In  the  fourth  step,  the  TCEA  will  estimate  the  annual  number 
of  students  who  must  be  trained  in  each  institutional  training 
course  in  order  to  sustain  the  new  system's  manpower 
requirements.  These  estimates  are  determined  by  calculating  how 
many  people  in  the  new  system's  projected  manpower  slots  can  be 
expected  to  leave  those  slots  each  year  based  on  projected 
attrition  and  promotion  rates. 


The 
time  for 
with  the 
system ' s 
the  TCEA 


fifth  step  will  produce  estimates  of  maximum  training 
the  initial  institutional  training  courses  associated 

notj  euef  cm  *  c  eruir  no  MOCc  /  f  ho  MOCc  f  t  m  uhi 


new  system's  source  MOSs  (the  MOSs  from  which  the  new 
operators  and  maintainers  will  be  drawn).  To  do  this, 
will  divide  the  available  training  days  by  the  required 
annual  student  load. 


The  sixth  step  will  produce  an  estimate  of  the  maximum 
amount  of  time  likely  to  be  available  to  train  the  new  system  in 
each  of  the  unit  types  that  are  expected  to  get  the  new  system. 
These  estimates  will  be  derived  from  information  contained  in  the 
Unit  Training  Time  Constraint  Data  Base. 


Steps  Involved  In  Estimating  Likely  Training  Time 


Steps  7  through 
maintainer  tasks  for 
likely  training  time 


12  will  generate  a  set  of  operator  and 
the  new  system  and  produce  estimates  of  the 
for  each  of  these  tasks  (see  Figure  2). 
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In  the  seventh  step,  the  user  will  generate  a  generic 
equipment  list  for  the  new  system.  To  do  this,  the  user  will  (1) 
review  the  list  of  major  equipment  items  associated  with  the 
replacement  system,  (2)  add  or  delete  items  from  this  list  to 
reflect  the  new  system's  mission,  functions,  and  performance 
requirements,  and  (3)  replace  references  to  specific  equipment 
items  with  references  to  generic  equipment  (e.g.,  replace 
reference  to  T999  engine  with  reference  to  diesel  engine). 

In  the  eighth  step,  a  maintenance  task  list  will  be 
generated  for  the  new  system.  The  TCEA  will  create  a  preliminary 
list  of  maintenance  task  statements  by  combining  the  equipment 
items  from  the  generic  equipment  list  with  task  actions  obtained 
from  the  AMMDB  for  similar  existing  equipment.  The  user  will 
then  have  the  opportunity  to  review  and  modify  these  task 
statements . 

In  the  ninth  step,  an  operator  task  list  will  be  generated 
for  the  new  system  by  linking  the  operational  tasks  produced  by 
Product  1  with  generic  equipment  items  identified  in  Step  7. 

In  the  tenth  step,  a  set  of  training  factor  ratings  will  be 
produced  for  each  new  system  task.  The  TCEA  will  assist  the  user 
in  identifying  a  similar  existing  task  that  has  training  factor 
data.  The  user  will  then  review  and  modify  these  training  factor 
data  to  reflect  the  new  system's  performance  requirements  and 
expected  equipment  design. 

In  the  eleventh  step,  the  TCEA  will  estimate  the  likely 
amount  of  time  that  will  be  spent  in  training  each  new  system 
task.  The  TCEA  software  will  apply  a  model  that  predicts 
training  time  as  a  function  of  the  nine  training  factors  the  Army 
currently  uses  to  determine  task  criticality. 

In  the  twelfth  step,  the  TCEA  will  estimate  the  amount  of 
time  tha  different  types  of  training  are  used  to  train  each  new 
system  task  in  both  institutional  and  unit  training.  An  on-line 
data  base  called  the  Training  Type  Allocation  Guidelines  will 
describe  the  estimated  percentage  of  time  that  each  type  of 
training  can  be  expected  to  be  used  in  a  mission  area.  Separate 
guidelines  will  be  provided  for  institutional  and  unit  training. 
The  TCEA  will  apply  these  percentages  to  the  total  training  time 
estimates  for  Step  12  to  produce  estimates  of  the  time  spent  on 
each  institutional  and  unit  training  type. 


Steps  Involved  in  Comparing  Maximum  and  Likely  Training  Time 

In  the  last  two  TCEA  steps,  maximum  and  likely  training  time 
are  compared,  and  the  TCEA  is  iterated  until  all  training  time 
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shortfalls  have  been  eliminated  (see  Figure  2).  In  the 
thirteenth  step,  the  system  will  compare  estimated  training  time 
to  the  maximum  training  time  constraint  and  identify  shortfalls 
(cases  where  the  estimated  training  time  exceeds  the  maximum 
available  training  time).  Separate  estimates  will  be  made  for 
the  initial  institutional  and  unit  training. 

In  the  fourteenth  step,  the  user  will  change  key  parameters 
and  iterate  the  TCEA  steps  until  training  time  shortfalls  have 
been  eliminated. 
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STEPS  IN  APPLYING  TCEA 


This  section  presents  a  discussion  of  the  steps  that  the 
analyst  will  take  in  applying  TCEA. 

The  discussion  of  each  step  is  organized  as  follows: 

1.  Input  -  the  data  that  are  required  for  the  completion 
of  this  step.  Input  data  consists  of  two  types: 

•  External  -  input  supplied  by  the  user  or  sources 
external  to  the  TCEA  or  other  (MPT)2  aids 

•  Internal  -  data  supplied  by  sources  within  the 
TCEA  or  other  (MPT)2  aids. 

2.  Process  -  the  substeps  that  the  analyst  will  apply 
during  this  step 

3.  Output  -  the  data  that  will  be  generated  as  a  result  of 
this  step 

4.  MA&D/DRC  Approach  for  Development  -  our  approach  for 
developing  the  automated  components  associated  with  the 
step 

There  are  fourteen  steps  in  applying  the  TCEA.  The  first 
six  steps  produce  estimates  of  maximum  training  time  for  initial 
institutional  and  unit  training.  The  new  six  steps  produce  an 
estimate  of  the  likely  training  time  for  each  generic  new  system 
task.  In  the  last  two  steps,  the  likely  training  times  are 
compared  to  the  maximum  training  times,  shortfalls  are 
identified,  and  the  TCEA  steps  are  iterated  until  shortfalls  are 
eliminated.  In  the  fourteenth  step,  the  user  can  make  changes  ^o 
key  parameters  and  iterate  the  first  thirteen  steps  until  these 
shortfalls  are  eliminated.  Figure  3  provides  an  overview  of  the 
first  six  steps  while  Figure  4  provides  an  overview  of  the  last 
eight  steps. 

We  have  purposely  repeated  information  in  Sections  4  and 

5.  We  hope  doing  this  will  make  it  easier  to  read  each  section 
independently . 


Step  1:  System/User  Obtain  Initial  Institutional  Training 

Course  Data 


Output 


During  this  step,  the  system  will  extract  institutional 
training  course  data  from  the  Army  Training  Requirements  and 
Resources  System  (ATRRS).  These  data  will  include  (a)  a  list  of 
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Figure  3.  Steps  Involved  In  Estimating  Maximum  Training  Time. 


Figure  4.  System/User  Iterates  Steps 


the  initial  institutional  training  courses  associated  with  the 
MOSs  from  which  the  operators  and  maintainers  of  the  new  system 
will  be  drawn,  (b)  the  length  of  these  courses,  and  (c)  the 
course  trainee  input  capacity.  The  course  trainee  input  capacity 
describes  the  maximum  amount  of  students  that  can  be  trained  in  a 
course  given  available  training  resources.  The  information 
obtained  in  this  step  will  be  used  in  subsequent  steps  to 
estimate  maximum  available  training  days  for  the  initial 
institutional  training  courses. 


Input 


External  Input.  The  user  will  be  able  to  examine  and  modify 
the  information  on  course  length  and  input  capacity  obtained  from 
ATRRS .  This  will  give  the  user  an  opportunity  to  assess  the 
impact  of  changes  in  these  two  variables  on  maximum  training 
time . 


Internal  Input.  Product  2  will  provide  a  listing  of  the 
source  MOSs  for  the  new  system.  These  MOSs  will  describe  the 
likely  MOS  populations  from  which  the  operators  and  maintainers 
of  the  new  system  will  be  drawn. 


Process 

The  user  will  log-on,  call  up  the  TCEA,  and  tell  it  to  begin 
the  analysis.  Using  the  MOS  list  produced  by  Product  2  as  a  key, 
the  system  will  call  up  and  search  through  the  Army  Training 
Requirements  and  Resources  System  (ATRRS)  to  identify  the  initial 
institutional  training  course  associated  with  each  MOS.  The 
system  will  use  a  modem  to  call  up  the  ATRRS.  Then  the  system 
will  enter  and  execute  a  data  extraction  program  on  the  ATRRS 
host  computer.  This  program  will  place  the  extracted  data  on  a 
flat  file  and  send  it  back  to  the  user's  computer.  It  may  take 
the  TCEA  a  few  minutes  to  make  these  data  extractions.  A  message 
will  appear  on  the  screen  which  tells  the  user  that  data 
extractions  are  being  made  and  which  estimates  when  they  will  be 
completed.  Once  the  data  extractions  have  been  made  the  user  can 
review  and  modify  the  data  on  course  length  and  trainee  input 
capacity . 


MA&D/DRC  Approach  for  Development 

All  software  for  the  TCEA  will  be  developed  in  accordance 
with  the  software  development  methodology  described  in  Appendix  A 
and  the  user  interface  guidelines  described  in  the  Product 
Requirements  Section  "Description  and  Development  of  Automated 
Components"  p.  74. 
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Development  of  ATRRS  Data  Extraction  Program.  The  program 
for  extracting  information  from  ATRRS  will  present  no  major 
technical  problems.  DRC  already  has  the  ATRRS  user’s  manuals  and 
the  ATRRS  data  element  description.  The  specific  data  elements 
for  which  we  will  extract  data  are  MOS  (data  element  E-YY), 
course  number  (CRS-NO),  course  title  (CRS-TITLE),  course  annual 
student/trainee  input  capacity  (ANN-CAP),  and  course  length  in 
days  (CRS-DAYS).  The  course  number  code  can  be  used  to  identify 
which  course  is  an  initial  training  course. 


Step  2:  Dser  Identifies  Portion  of  Each  Course  Devoted 

to  Other  Systems 


Output 

Many  initial  training  courses,  particularly  those  for 
maintenance  MOSs,  provide  training  on  a  wide  range  of  systems. 
During  this  step,  the  user  will  examine  the  initial  training  POIs 
and  identify  the  time  spent  on  training  other  weapon  systems  that 
are  still  operational  when  the  new  system  is  fielded.  This 
training  time  is  assumed  to  be  a  fixed  constraint  which  the  new 
system  cannot  change.  In  Step  5,  this  training  time  will  be 
subtracted  from  the  estimated  initial  training  course  length  to 
produce  an  estimate  of  the  time  that  will  be  spent  in  training 
the  new  system  in  the  initial  training  course. 


I  npu  t 

External  Input.  The  user  must  examine  the  initial  training 
POIs  to  determine  the  amount  of  time  devoted  to  training 
different  systems. 

Internal  Input.  Product  2  will  provide  a  list  of  the 
systems  which  will  be  repla.ed  by  the  new  system.  An  on-line 
data  base,  called  the  New  System  Fielding  Schedule,  will  describe 
the  major  new  weapon  systems  expected  to  be  fielded  in  each 
mission  area,  the  systems  they  will  replace,  and  the  date  they 
are  expected  to  be  fielded. 


Process 


The  system  will  produce  a  hard  copy  report  listing  (a)  the 
initial  training  courses  associated  with  each  new  system  source 
MOS,  (b)  the  existing  systems  which  the  new  system  will  replace, 
and  (c)  the  New  System  Fielding  Schedule.  Using  this  report  as 
input,  the  user  will  review  the  initial  institutional  training 
POI  and  describe  the  amount  of  time  spent  on  training  other 
weapon  systems  that  are  still  operational  when  the  new  system  is 
f ielded . 
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MA&D/DRC  Approach  for  Development 


Development  of  New  System  Fielding  Schedule.  The  New  System 
Fielding  Schedule  will  describe  the  major  new  weapon  systems 
which  are  expected  to  be  fielded  in  each  mission  area,  the 
systems  they  will  replace,  and  the  date  they  are  expected  to  be 
fielded.  All  of  the  information  needed  to  produce  the  New  System 
Fielding  Schedule  is  available  in  the  Army  Modernization 
Improvement  Memorandum.  We  will  simply  review  this  data  and 
place  it  in  the  data  base. 


Step  3:  System  Estimates  Maximum  Available  Training  Days 
Output 

During  this  step,  the  system  will  estimate  the  maximum 
available  training  mandays  for  each  institutional  training 
course.  This  will  be  accomplished  using  the  following  algorithm: 


Maximum 

Current 

Course 

Trainee 

Available 
Tr.  Days 

=  Course 
Length 

X 

Input 

Capacity 

Information  on 

course  length 

and 

course 

trainee 

capacity  will  be  obtained  from  ATRRS  during  Step  1.  The  course 
trainee  input  capacity  describes  the  maximum  amount  of  students 
that  can  be  trained  in  a  course  c  ven  available  training 
resources. 


Input 

External  Input.  None.  This  step  is  completely  automated. 

Internal  Input.  Step  1  will  provide  information  on  course 
length  and  course  trainee  input  capacity  for  all  initial 
institutional  training  courses  associated  with  the  new  system. 


Process 

The  system  will  simply  multiply  course  length  by  trainee 
input  capacity  to  produce  an  estimate  of  the  maximum  available 
training  mandays  for  initial  institutional  training. 


MA&D/DRC  Approach  for  Development 

Development  of  the  software  associated  with  this  step  will 
present  no  major  technical  problems. 
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Step  4:  System  Determines  Required  Annual  Student  Load 


Output 

The  output  of  this  step  is  estimates  of  the  annual  number  of 
students  who  must  be  trained  in  each  institutional  training 
course  in  order  to  sustain  the  new  system  manpower  requirements. 
These  estimates  are  determined  by  calculating  how  many  people  in 
the  new  system's  projected  manpower  slots  can  be  expected  to 
leave  those  slots  each  year.  The  training  system  must  train 
people  to  replace  them. 


Input 


External  Input .  The  user  will  have  the  option  of  either 
using  the  maximum  manpower  constraints  produced  by  Product  1  as 
an  estimate  of  the  manpower  requirements  for  the  new  system,  or 
inputting  his  or  her  own  estimate  of  these  requirements. 
Potential  data  sources  for  manpower  requirements  estimates 
include  MAA  results  and  the  results  of  feasibility  studies 
conducted  during  the  Requirements/Technology  Base  Activities 
Phase.  The  user  may  also  want  to  simply  input  a  "best  guess"  of 
manpower  requirements.  Users  will  be  encouraged  to  use  the 
manpower  constraints  as  the  value  for  manpower  requirements, 
since  it  will  produce  a  worst-case  estimate  of  maximum  training 
time . 


Internal  Input.  Product  2  will  provide  a  list  of  the  source 
MOSs  and  paygrades  for  the  new  system.  Product  3  will  provide  an 
estimate  of  the  projected  attr ition/promotion/MOS  migration  rates 
for  these  same  MOSs. 


Process 


Figure  5  provides  an  overview  of  the  process  for  this 
step.  The  user  begins  by  selecting  which  option  he  or  she  will 
use  to  define  manpower  requirements  for  the  new  system  —  the 
maximum  manpower  constraints  from  Product  2  or  the  user’s  own 
estimate  of  these  requirements.  If  the  user  selects  the 
constraint  option,  the  system  will  extract  the  maximum  manpower 
constraints  from  output  files  created  by  Product  2.  If  the  user 
decides  to  enter  his  or  her  own  estimates  of  manpower 
requirements,  he  or  she  will  be  presented  with  an  input  form 
listing  the  MOSs  and  paygrades  that  require  manpower 
requirements.  (These  will  be  determined  by  Product  2.) 

Once  manpower  requirements  have  been  determined,  the  system 
will  determine  the  estimated  annual  loss  rate  for  paygrades  1  to 
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Figure  5.  Overview  of  Logic  for  Determining  Annual  Required  Student  Load 


3  for  each  MOS  by  extracting  this  information  from  files  created 
by  Product  3.  The  system  will  then  calculate  the  combined  loss 
rate  for  these  paygrades  1  to  3.  Loss  rates  are  calculated  for 
personnel  in  these  paygrades  only  since  these  are  the  personnel 
whose  replacements  must  be  trained  in  initial  institutional 
training.  Of  course  this  approach  assumes  that  the  grade 
structure  for  an  MOS  has  been  developed  in  accordance  with 
approved  standards  of  grade.  This  insures  that  there  will  be  the 
proper  number  of  people  at  the  lower  paygrades  to  feed  the 
manpower  slot  at  the  upper  paygrades.  The  manpower  constraints 
produced  by  our  Product  2  will  be  developed  in  accordance  with 
the  standards  of  grade. 

In  the  final  substep,  the  system  will  calculate  the  required 
annual  load  for  each  initial  institutional  training  course  by 
multiplying  the  manpower  requirements  for  paygrades  1  to  3  in  an 
MOS  by  the  expected  annual  loss  rate  for  those  paygrades. 


MA&D/DRC  Approach  for  Development 

Development  of  Projected  Loss  Rates.  We  assume  that 
projected  loss  rates  for  each  of  the  new  system's  source  MOSs  and 
paygrades  will  be  determined  by  Product  3.  Product  3  will  use 
these  rates  to  develop  estimates  of  the  expected  future 
distributions  of  key  personnel  characteristics  within  each  source 
MOS.  If  Product  3  does  not  produce  these  loss  rates,  we  propose 
to  use  the  existing  attrition  and  promotion  rates  in  the 
calculation  of  student  load.  Information  on  current  loss  rates 
can  be  obtained  directly  from  DMDC  data  files  or  from  the 
FORECAST  system  data  bases.  DRC  already  receives  tape  extracts 
of  DMDC  loss  files  and  has  developed  several  programs  for 
extracting  data  from  them. 


Step  5:  System  Calculates  Maximum  Training  Time  for 

Initial  Training 


Output 


This  step  will  produce  estimates  of  maximum  training  time 
for  the  initial  institutional  training  courses  associated  with 
the  new  system's  source  MOSs  {that  is,  the  MOSs  from  which  the 
operators  and  maintainers  of  the  new  system  will  be  drawn). 
These  estimates  will  be  determined  by  applying  the  following 
simple  equation: 


Maximum 

Training 

Time 


Available 

Training 

Days 


Required  Annual 

Student 

Load 
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Input 


External  Input.  None.  This  step  will  be  completely 
automated . 

Internal  Input .  Step  3  will  provide  input  on  the  maximum 
available  training  days  for  the  institutional  training  courses. 
Step  4  will  provide  input  on  the  annual  student  load  required  to 
support  the  new  system.  Step  2  will  provide  input  on  the  amount 
of  time  that  must  be  spent  on  training  other  systems  in  the 
source  MOS  institutional  training  courses. 


Process 


The  system  will  divide  the  available  training  days  by  the 
required  annual  student  load  to  produce  an  initial  estimate  of 
maximum  training  time  for  each  course.  The  time  required  to 
train  other  weapon  systems  (produced  by  Product  2)  will  be 
subtracted  from  the  initial  estimate  to  provide  an  estimate  of 
the  total  amount  of  time  that  will  be  spent  in  training  the  new 
system. 


MA&D/DRC  Approach  for  Development 

Development  of  the  software  associated  with  this  step  will 
present  no  technical  problems. 


Step  6:  System  Estimates  Maximum  Unit  Training  Times 


Output 

This  step  will  produce  an  estimate  of  the  maximum  amount  of 
time  available  to  train  the  new  system  in  each  of  the  unit  types 
expected  to  get  the  new  system.  The  user  can  break  out  separate 
estimates  for  the  locations  (i.e.,  CONUS,  Europe,  etc.)  expected 
to  get  the  new  system. 


Input 


External  Input.  If  the  user  wants  separate  training  time 
estimates  for  different  locations,  he  or  she  must  identify  these 
locations  and  indicate  which  unit  types  will  be  assigned  to  each 
location . 

Internal  Input.  Product  2  will  provide  a  list  of  the  types 
of  units  that  will  get  the  new  system,  the  maximum  crew  size,  and 
the  number  of  crews  per  unit  type. 
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Three  on-line  data  bases  will  provide  input  to  this  step. 

The  Unit  Training  Time  Constraint  data  base  will  list  the  maximum 
amount  of  time  that  different  types  of  units  are  expected  to  have 
to  devote  to  weapon  system-specif ic  training.  The  estimates  will 
be  broken  out  by  mission  area  and  by  unit  location. 

The  New  System  Fielding  Schedule,  developed  in  Step  2,  will 
describe  the  major  new  weapon  systems  expected  to  be  fielded  in 
each  mission  area,  the  systems  they  will  replace,  and  the  date 
they  are  expected  to  be  fielded. 

The  Major  System  by  Unit  Data  Base  will  list  the  major 
systems  associated  with  different  unit  types. 


Process 


Figure  6  provides  an  overview  of  this  process.  The  system 
will  present  the  user  with  a  list  of  the  type  of  units  to  get  the 
new  system.  The  user  will  indicate  whether  he  or  she  wants  to 
break  out  the  unit  training  time  estimates  by  location.  If  so, 
the  user  will  list  the  locations  associated  with  each  unit.  If 
not,  or  if  all  required  unit  location  information  has  been 
entered,  the  user  will  estimate  the  number  of  different  weapon 
systems  that  will  be  trained  in  the  units  getting  the  new 
system.  The  user  can  examine  the  New  System  Fielding  Schedule, 
which  will  describe  the  major  new  weapon  systems  which  will  be 
fielded  in  each  mission  area  and  the  systems  they  will  replace, 
and  the  Major  System  by  Unit  Data  Base  which  will  list  the  major 
systems  associated  with  different  unit  types. 


In  the  final  substep,  the  system  will  extract  data  from  the 
Unit  Training  Time  Constraint  Data  Base,  which  will  describe  the 
maximum  amount  of  time  that  can  be  spent  on  system-specific 
training  for  the  unit  types  getting  the  new  system.  The  system 
will  then  divide  this  time  by  the  estimated  number  of  weapon 
systems  to  be  trained  in  those  unit  types.  This  will  produce 
estimates  of  the  maximum  total  amount  of  time  that  can  be  spent 
in  unit  training  for  the  new  system  for  all  students.  This 
system  will  then  multiply  the  maximum  crew  size  by  number  of 
crews  per  unit  type.  (Both  of  these  values  will  be  supplied  by 
Product  2.)  This  will  produce  an  estimate  of  the  total  number  of 
soldiers  who  must  be  trained  in  the  unit.  This  number  will  be 
divided  into  the  maximum  total  amount  of  time  for  all  students  to 
produce  an  estimate  of  maximum  time  per  student. 


MA&D/DRC  Approach  for  Development 

Development  of  the  Unit  Training  Time  Constraint  Data  Base. 
The  Unit  Training  Time  Constraint  Data  Base  will  list  the  maximum 
amount  of  time  that  different  types  of  units  are  expected  to 
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Figure  6.  Overview  of  Logic  for  Estimating  Maximum 
Unit  Training  Times. 


devote  to  weapon  system-specif ic  training.  The  estimates  will  be 
broken  out  by  mission  area  and  by  unit  location. 

Unfortunately,  the  information  needed  to  complete  the  Unit 
Training  Time  Constraint  Data  Base  is  not  available  in  any 
automated  data  base.  To  develop  this  information,  we  propose  to 
send  out  a  survey  questionnaire  to  a  representative  sample  of 
unit  training  managers.  The  survey  will  provide  data  for  several 
of  the  TCEA  steps.  Table  7  lists  the  types  of  questions  which 
will  be  included  in  the  survey,  and  the  TCEA  steps  which  will  use 
the  survey  data. 

The  survey  will  be  developed  in  accordance  with  state-of- 
the-art  guidelines  for  mail  surveys  (Altschuld  &  Lower,  1984; 
Dillman,  1978;  Dyer,  et  al.,  1976;  Eudos,  1983;  Sudman  & 

Bradburn,  1982;  Lockhart,  1984).  DRC  has  successfully  conducted 
a  number  of  surveys  for  ARI.  These  surveys  include  studies  on 
the  Joint  Optical  Information  Network  (JOIN)  (Lockhart,  1987), 
Analysis  of  the  Soldier  in  Europe  Survey  Data  (Lockhart,  1987), 
and  the  user  survey  for  the  Man  Integrated  System  Technology 
(MIST)  (Dynamics  Research  Corporation,  1985). 

We  will  identify  the  type  of  units  that  operate  and  maintain 
major  weapon  systems  and  select  a  stratified  sample  of  battalion 
unit  training  managers  from  each  major  command. 

To  obtain  a  high  response  rate,  we  will  mail  pre-notices, 
questionnaires,  and  follow-up  items  to  the  participants.  In  a 
recent  DRC  survey  (Lockhart,  et  al.,  1987)  we  used  a  series  of 
five  mailings  to  increase  the  number  of  respondents.  This 
mailing  schedule  is  outlined  in  Table  8. 

The  pre-notice  informs  the  participants  that  the  question¬ 
naire  is  coming  and  asks  them  for  their  assistance.  The  survey 
is  mailed  out  the  week  after  the  pre-notice.  A  reminder  asks  all 
participants  to  complete  the  survey  and  provides  a  toll-free 
number  they  can  use  for  help.  A  second  questionnaire  is  sent  to 
soldiers  who  have  not  returned  the  first  questionnaire  within 
four  weeks.  A  thank  you  letter  is  sent  to  each  participant  after 
receipt  of  a  completed  questionnaire. 

Development  of  the  Major  System  by  Unit  Data  Base.  The  Major 
System  by  Unit  na*-“  will  list  the  major  systems  associated 

with  different  unit  types.  A  iist  of  the  major  systems 
associated  with  a  unit  can  be  obtained  from  the  TOE  for  that 
unit . 
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Table  7.  Type  of  Questions  to  Be  Included  in  Unit  Training  Time  Survey. 
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•These  are  not  the  actual  questions  that  will  be  Included  In  the  survey.  They  are  the  topics  that  the  questions  win  cover. 


Table  8.  Proposed  Mailing  Schedule  for  Unit  Training 

Time  Survey. 


Mailing  Date 

Pre-notice  One  Week  Prior  to  Survey 

Survey 

Reminder  One  Week  After  Survey 

Second  Questionnaire  Four  Weeks  After  Survey 

Thank  You  After  Return  of  Survey 
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Step  7:  User  Generates  New  System  Generic  Equipment  List 


Output 

The  output  of  this  step  will  be  (1)  a  generic  equipment  list 
for  the  new  system,  and  (2)  a  list  of  the  existing  equipment  most 
similar  to  the  new  system.  By  constructing  the  generic  equipment 
list,  users  will  be  allocating  operational  functions  to  equipment 
types.  This  must  be  done  before  creating  task  statements  for  the 
new  system.  The  list  of  similar  equipment  will  help  the  user 
identify  existing  training  factor  ratings  in  the  Training  Factor 
Ratir;  Data  Base. 


Input 


External  Input.  The  user  will  generate  the  generic 
equipment  list  and  the  list  of  similar  equipment. 

Internal  Input.  Product  1  will  provide  a  list  of  the  new 
system  missions,  functions,  and  performance  requirements. 

Product  2  will  provide  a  list  of  the  systems  to  be  replaced  by 
the  new  system. 

An  on-line  data  base,  called  the  Equipment  by  Major  System 
Data  Base,  will  also  provide  input  to  this  step.  This  data  base 
will  list  the  major  equipment  items  associated  with  Army  weapon 
systems  and  the  items  included  in  the  Training  Factor  Rating  Data 
Base. 


Process 


Figure  7  provides  an  overview  of  the  process  for  this 
step.  The  system  will  begin  by  constructing  a  list  of  the  major 
equipment  items  of  the  systems  being  replaced.  Product  2  will 
provide  a  list  of  the  systems  being  replaced.  The  Equipment  by 
Major  System  Data  Base  will  list  the  major  equipment  items 
associated  with  existing  Army  weapon  systems.  The  system  will 
simply  access  this  data  base  using  the  Product  2  replacement 
system  list  as  a  key. 

In  the  second  substep,  the  user  will  generate  a  generic 
equipment  list  for  the  new  system.  The  user  will  (1)  review  the 
list  of  major  equipment  items  associated  with  the  replacement 
system,  (2)  add  or  delete  items  from  this  list  to  reflect  the  new 
system's  mission,  functions,  and  performance  requirements,  and 
(3)  replace  references  to  specific  equipment  with  references  to 
generic  equipment  (e.g.,  replace  reference  to  T999  engine  with 
reference  to  diesel  engine).  The  user  will  also  identify  the 
replacement  system  equipment  items  associated  with  each  generic 
equipment  item. 
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Figure  7.  Overview  of  Logic  for  Generating  Generic  Equipment  List 


In  the  third  substep,  the  user  will  identify  the  existing 
equipment  items  which  are  expected  to  be  most  similar  to  the  new 
system's  generic  equipment  items  in  terms  of  time  needed  to  train 
operators  and  maintainers  and  which  have  available  training 
factor  data.  To  assist  the  user,  the  system  will  identify  the 
replacement  items  associated  with  the  generic  equipment  items, 
and  identify  which  of  these  replacement  items  have  data  in  the 
Training  Factor  Rating  Data  Base.  The  user  will  also  be  able  to 
query  the  Equipment  by  Major  System  Data  Base  to  obtain  a  list  of 
additional  equipment.  Each  equipment  item  in  the  data  base  will 
be  tied  to  the  major  system(s)  in  which  it  is  embedded,  and  the 
major  systems  will  be  grouped  by  mission  area. 


MA&D/DRC  Approach  For  Development 

Development  of  the  Equipment  by  Major  System  Data  Base. 

This  data  base  will  list  the  major  equipment  items  associated 
with  the  major  weapon  systems  in  each  mission  area.  We  expect 
about  ten  to  fifteen  major  equipment  items  per  weapon  system. 
Lists  of  the  major  equipment  items  associated  with  a  weapon 
system  can  be  found  in  several  sources  including  the  AMIM  and  the 
MMMDB.  ( DRC  has  on-line  access  to  the  AMMDB. ) 


Step  8:  System  Generates  Maintenance  Task  List  for  New  System 

Output 

This  step  will  produce  a  maintenance  task  list  for  the  new 
system.  Product  1,  the  SPREA,  will  provide  information  on 
overall  system  performance  requirements  for  both  operations  and 
maintenance.  The  SPREA  will  estimate  the  low-level  operational 
functions  associated  with  a  system's  mission.  It  is  expected 
that  these  low-level  operational  functions  can  be  mapped  onto 
operator  tasks  on  a  one-to-one  basis.  However,  we  believe  that 
Product  1  will  not  be  able  to  produce  an  estimate  of  functions  at 
the  maintenance  task  level.  It  is  difficult  to  identify 
maintenance  tasks  until  operational  functions  have  been  allocated 
to  particular  types  of  equipment.  The  performance  requirements 
for  a  maintenance  task,  indeed  the  maintenance  tasks  themselves, 
will  depend  on  what  it  is  that  must  be  maintained.  Product  1 
must  estimate  functional  performance  requirements  without  any 
function  allocation.  Thus,  in  this  context,  it  is  unlikely  that 
Product  1  can  estimate  maintenance  tasks. 

In  the  previous  step,  a  preliminary  function  allocation  was 
made  to  generic  equipment  items,  and  these  generic  items  were 
linked  to  existing  equipment  items  with  established  maintenance 
tasks.  In  this  step,  the  system  will  access  the  AMMDB  to  obtain 
a  listing  of  these  maintenance  tasks.  The  TCEA  will  generate  a 
preliminary  maintenance  task  list  for  the  new  system  by 
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allocating  maintenance  tasks  from  the  existing  equipment  to  the 
new  system  generic  equipment  items. 


Input 


External  Input.  The  user  will  be  able  to  review  and  modify 
the  new  system  maintenance  task  list. 

The  system  will  access  the  Army's  fielded  maintenance  data 
collection  system  to  generate  a  list  of  the  maintenance  tasks 
associated  with  the  equipment  items  most  similar  to  the  new 
system. 

Internal  Input.  Step  8  will  provide  a  list  of  the  existing 
equipment  items  which  are  most  similar  to  the  new  system. 


Process 


The  system  will  begin  by  obtaining  a  complete  list  of  the 
maintenance  tasks  for  existing  equipment  items  which  are  most 
similar  to  the  new  system.  The  system  will  use  a  modem  to  call 
up  the  AMMDB .  Then  the  system  will  enter  and  execute  a  data 
extraction  program  on  the  AMMDB  host  computer.  The  program  will 
place  the  extracted  data  on  a  flat  file  and  send  it  back  to  the 
user's  computer.  It  may  take  the  PREA  an  hour  or  more  to  make 
these  data  extractions.  A  message  will  appear  on  the  screen 
estimating  when  they  will  be  completed. 

The  system  will  create  a  preliminary  list  of  maintenance 
tasks  for  the  new  system  by  constructing  task  statements  that 
combine  equipment  items  from  the  generic  equipment  list  with  task 
actions  from  similar  existing  equipment.  The  user  will  be  able 
to  review  and  modify  these  task  statements. 


MA&D/DRC  Approach  for  Development 

Development  of  Program  for  Extracting  AMMDB  Data.  As  part 
of  Product  2,  we  will  develop  a  program  for  extracting 
maintenance  manhours  information  from  the  AMMDB.  This  program 
will  extract  total  maintenance  hours  performed  by  a  particular 
MOS  and  grade  on  a  particular  piece  of  equipment.  The  TCEA  needs 
information  one  step  below  this  level  —  that  is,  it  needs 
information  on  the  individual  maintenance  tasks  associated  with 
these  maintenance  hours.  This  information  is  readily  available 
in  the  AMMDB.  We  see  no  major  technical  difficulties  associated 
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with  modifying  the  Product  2  program  to  extract  this  additional 
data  or  constructing  a  new  program  should  one  not  be  created  as 
part  of  Product  2.  DRC  already  has  on-line  access  to  the  AMMDB 
and  has  many  programs  for  extracting  data  from  this  data  base. 


Step  9:  User/System  Generates  Operator  Task  List 


Output 

This  step  will  produce  a  complete  set  of  operator  task 
statements  for  the  new  system.  Product  1  will  produce  a  list  of 
operational  functions  expected  to  be  the  same  as  operator  task 
actions.  However,  to  develop  a  complete  task  statement  these 
task  actions  must  be  linked  to  equipment.  Step  9  will  make  these 
linkages . 


I  nput 


External  Input.  The  user  will  link  the  operational  tasks 
from  Product  1  to  the  generic  equipment  items  identified  in  Step 
1. 


Internal  Input.  Product  1  will  provide  a  list  of  the 
operational  tasks  expected  to  be  associated  with  the  new 
system.  Product  2  will  provide  a  list  of  the  source  MOSs  for  the 
new  system.  Step  1  will  provide  a  list  of  the  generic  equipment 
items  expected  to  be  associated  with  the  new  system. 


Process 

The  system  will  first  present  the  user  with  a  list  of  the 
operational  tasks  from  Product  1  and  ask  the  user  to  indicate 
which  of  these  tasks  will  be  performed  by  humans  and  if  they  will 
perform  them.  The  system  will  then  present  the  user  with  an 
operational  task  and  a  menu  displaying  generic  equipment  items. 
The  user  will  indicate  which  items  will  be  used  to  perform  the 
operational  task.  The  system  will  construct  an  operator  task 
list  by  combining  the  operational  tasks  and  the  selected  generic 
equipment  items.  The  list  will  be  presented  to  the  user  for 
review  and  update. 


KA&D/DPC  Approach  for  Development 

We  will  create  some  simple  software  to  combine  task  actions 
and  equipment  items  into  task  statements. 
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Step  10:  User  Determines  Training  Factor  Ratings 
for  New  System  Tasks 


Output 

The  system  will  produce  a  set  of  training  factor  ratings  for 
each  new  system  task.  The  system  will  identify  similar  existing 
tasks  with  existing  training  factor  data.  The  user  will  review 
and  modify  these  ratings  to  reflect  the  new  system's  performance 
requirements  or  expected  equipment  design. 


Input 


External  Input.  The  system  will  automatically  identify 
similar  tasks.  On  some  occasions,  however,  the  user  may  have  to 
assist  the  system  in  identifying  the  similar  tasks. 

Internal  Input.  Product  1  will  provide  a  description  of  the 
new  system's  performance  requirements.  Step  7  will  provide  a 
list  of  the  equipment  items  which  are  expected  to  be  most  similar 
to  the  new  system  equipment  and  the  new  system's  generic 
equipment  list.  Step  8  will  provide  a  complete  maintenance  task 
list  for  the  new  system.  Step  9  will  provide  a  complete  operator 
task  list  for  the  new  system. 

Four  on-line  data  bases  will  provide  input  to  this  step. 
Training  factor  ratings  for  the  tasks  associated  with  existing 
systems  will  be  stored  in  the  Training  Factor  Rating  Data  Base. 
The  data  base  will  contain  the  ratings  from  past  analyses  of 
these  tasks.  A  Verb  Thesaurus  will  identify  synonyms  for  common 
task  actions,  a  Task  Taxonomy  will  group  together  action  verbs 
based  on  similarities  in  training  factor  ratings.  An  Equipment 
Item  Taxonomy  will  group  together  equipment  items  based  on 
similarities  in  the  training  factor  ratings  for  their  associated 
tasks . 

Process 


Figure  8  provides  an  overview  of  the  process  for  this 
step.  The  user  can  either  use  training  factor  ratings  from 
similar  existing  tasks  to  quide  his  or  her  ratings,  or  enter  the 
training  factor  ratings  fc .  the  new  system  directly  without  using 
baseline  information  from  similar  tasks.  If  the  user  decides  to 
use  the  baseline  information,  the  system  will  begin  by  selecting 
a  task  from  the  new  system  operator  or  maintainer  task  list.  The 
system  will  identify  the  similar  equipment  associated  with  this 
task  in  Step  1.  It  will  then  examine  the  task  actions  in  the 
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Training  Factor  Rating  Data  Base  associated  with  this  equipment 
and  compare  them  to  the  new  system  task  action  verb.  If  one  of 
these  actions  matches  the  verb,  the  system  will  extract  the 
training  factor  ratings  for  the  task  and  temporarily  assign  them 
to  the  new  system  task.  If  there  is  not  a  match,  the  system  will 
use  the  Verb  Thesaurus  to  find  a  synonym.  If  there  is  no 
synonym,  the  user  will  be  asked  to  pick  a  similar  task  from  the 
Training  Factor  Rating  Data  Base.  The  user  will  be  able  to 
examine  the  Equipment  Taxonomy  and  the  Task  Taxonomy. 

The  user  can  review  the  training  factor  ratings  associated 
with  the  similar  tasks  and  update  them  to  reflect  the  new 
system's  equipment  design  and  system  performance  requirements. 


MA&D/DRC  Approach  For  Development 

Data  for  Training  Factor  Data  Base.  All  of  the  data  in  the 
Training  Factor  Rating  Data  Base  will  be  obtained  from  the  CODAP 
data  base,  which  is  part  of  the  Army  Occupational  Survey  Program 
( AOSP ) .  We  will  request  a  tape  extract  containing  results  from 
all  of  the  training  factor  surveys  conducted  by  AOSP  since  1981, 
the  year  when  the  surveys  were  initiated.  We  will  load  into  the 
TCEA  Training  Factor  data  base  only  those  factors  which  are  part 
of  our  training  time  prediction  model  (that  is,  those  factors 
which  are  significant  predictors  of  institutional  or  unit 
training  time).  See  the  section  entitled  "MA&D/DRC  Approach  for 
Development,"  pp.  64-66  for  a  detailed  description  of  our 
approach  to  developing  this  model. 

Our  approach  does  not  require  that  the  Training  Factor  Data 
Base  contain  task  factor  ratings  for  all  Army  tasks.  All  that  we 
need  are  ratings  for  a  fairly  large  number  of  tasks.  The  purpose 
of  the  Training  Factor  Data  Base  is  to  provide  the  user  with  some 
baseline  data  from  a  similar  task.  If  data  are  not  there  for  one 
particular  task,  the  user  can  identify  another  task  that  does 
have  existing  data. 

Development  of  Task  Verb  Thesaurus.  There  are  several 
microcomputer-based  data  base  management  systems  which  already 
have  front-end  natural  language  processors.  If  we  use  one  of 
these  data  bases  to  store  the  Training  Factor  Rating  Data  Base, 
we  should  be  able  to  use  its  processor  to  identify  verb 
synonyms.  If  we  don’t  use  one  of  these  data  bases,  a  simple 
thesaurus  of  approximately  100  verbs  can  be  created. 

Development  of  Task  Taxonomy.  The  task  taxonomy  will  group 
together  task  actions  based  on  similarities  in  their  training 
factor  ratings.  To  create  this  taxonomy,  mean  scores  on  each 
training  factor  measure  will  be  created  for  each  task  action  by 
averaging  across  all  tasks  which  use  that  action  in  the  data 
base.  Profile  difference  scores  between  the  actions  on  the  mean 
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training  factor  ratings  will  then  be  created.  These  profile 
difference  scores  will  provide  a  measure  of  the  similarity  of  one 
action  to  another.  The  profile  differences  scores  will  be 
entered  into  a  cluster  analysis  routine  which  will  produce  the 
task  action  clusters  which  will  form  the  taxonomy. 


Table  9  summarizes  some  of  the  different  clustering 
techniques  that  could  be  used  in  this  analysis.  We  will  select 
one  of  these  techniques  during  development  of  the  detailed  design 
specifications . 


Development  of  Equipment  Taxonomy.  The  equipment  taxonomy 
will  group  together  equipment  items  based  on  similarities  in  the 
equipment  items.  To  create  this  taxonomy,  mean  scores  on  each 
training  factor  measure  will  be  created  for  each  equipment  by 
averaging  across  all  the  tasks  associated  with  that  equipment 
item  in  the  data  base.  Profile  difference  scores  between  the 
equipment  items  on  the  mean  training  factor  ratings  will  then  be 
created.  These  profile  difference  scores  will  provide  a  measure 
of  the  similarity  of  one  equipment  item  to  another.  The  profile 
differences  scores  will  be  entered  into  a  cluster  analysis 
routine  which  will  produce  the  equipment  item  groups  which  will 
form  the  taxonomy. 


Step  11:  System  Estimates  Likely  Training  Time 


Output 

The  output  of  this  step  will  be  an  estimate  of  the  time  to 
train  each  new  system  task.  Separate  estimates  will  be  provided 
for  initial  institutional  training  (i.e.,  AIT)  and  for  unit 
training . 


I  H  n  n  t 


External  Input.  None.  This  step  will  be  completely 
automated . 

Internal  Input.  Step  10  will  provide  a  set  of  task 
criticality  ratings  for  each  task. 


Process 

A  Training  Time  Production  Model  will  contain  a  set  of 
regression  equations  which  will  predict  training  time  as  a 
function  of  the  training  factor  ratings.  Separate  equations  will 
predict  initial  institutional  and  unit  training  time.  The 
initial  training  time  equation  will  estimate  the  time  which  is 
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Table  9.  Potential  Clustering  Techniques. 
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likely  to  be  spent  training  the  task  during  the  initial 
institutional  training.  The  unit  training  equation  will  estimate 
the  annual  time  spent  training  a  task  in  a  unit.  Each  mission 
area  will  have  a  separate  set  of  prediction  equations. 


MA&D/DRC  Approach  for  Development 

To  develop  the  training  time  prediction  equations,  we  will 
conduct  a  series  of  regression  analyses  using  the  training 
factors  as  predictors  and  the  time  to  train  a  task  as  a 
criterion.  We  will  develop  equations  to  predict  institutional 
and  unit  training  time  within  each  mission  area. 

To  develop  the  data  for  the  regression  analysis,  we  will 
select  a  sample  of  tasks  for  each  mission  area  from  the  CODAP 
data  base.  We  will  extract  data  on  the  nine  training  factors 
from  this  data  base.  Table  10  describes  the  scales  used  to 
obtain  ratings  for  each  of  the  factors.  The  average  percent  time 
performing  ratings  are  completed  by  enlisted  personnel  as  part  of 
their  basic  survey.  As  part  of  this  survey,  these  personnel 
indicate  which  tasks  they  perform.  The  percent  members 
performing  score  is  derived  from  an  analysis  of  this  data. 

Ratings  on  the  seven  other  training  factors  are  provided  by 
senior  NCOs  in  a  supplementary  survey. 

We  will  obtain  data  on  the  time  spent  in  training  the  tasks 
in  initial  training  by  examining  the  Program  of  Instruction  for 
the  initial  institutional  training  course.  (This  course  can  oe 
identified  by  examining  the  Trainer's  Guide  for  the  MOS 
associated  with  the  task.) 

We  will  obtain  er-imates  of  the  annual  amount  of  time  spent 
in  unit  training  from  _  survey  questionnaire  sent  to  unit 
training  managers.  We  will  use  this  same  survey  to  determine  the 
maximum  time  available  to  train  all  system  tasks.  See  the 
section  entitled  "Step  6:  System  Estimates  Maximum  Unit  Training 
Times",  p.  51  for  a  more  detailed  description  of  the  survey 
procedures . 

Once  we  have  collected  data  on  training  time,  we  will 
conduct  a  series  of  step-wise  regression  analyses  using  the 
training  factor  variables  as  predictors  and  the  unit  and 
institutional  training  time  measures  as  criterion  variables.  In 
conducting  the  regression  analyses,  we  will  identify  the  minimum 
number  of  training  factor  variables  needed  to  predict  training 
time  for  each  mission  area.  By  minimizing  the  number  of  training 
factor  variables,  we  can  reduce  the  number  of  ratings  that  users 
must  provide  in  Step  10.  Several  studies  indicate  that  the  nine 
training  factors  are  redundant  and  can  be  reduced  to  a  smaller 
number . 
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Table  10.  Training  Factor  Rating  Scales. 


Training  Factor 

Definition 

Scale 

Source  of 
Rating 

Percent  of  members 
performing 

Perce-  of  )ob  Incumbents 
who  perform  the  task 

Percent  scale  based  upon 
proportion  of  Yea/No 
responses 

Self-Report 

(Enlisted) 

Average  percent  of 
time  spent  by  members 
performing 

Average  percent  time  spent 
on  the  task  ratatfvs  to  time 
spent  on  all  othar  tasks 

7  -  Point  Relevant  Time 

Spent  scale 

(1  -  Very  much  below  average  to 

7  -  Very  much  above  average) 

Self-Report 

(Enlisted) 

Task  Learning 

Difficulty 

The  amount  of  tlma  required 
to  learn  to  perform  the  task 
satisfactorily 

7 -Point  scale 
(1  •  Extremely  low  learning 
difficulty  to 

7  ■  Extremely  high  (earning 
difficulty) 

Supervisory 

Ratings 

(NCOs) 

Consequences  of 
Inadequate 

Performance 

The  consequences  of  performing 
the  task  Inadequately  upon 
personnel  Injury,  loss  of  Ilfs, 
equipment  damege 

7-  Point  scale 

(1  -  Extremely  low  consequence  to 

7  -  Extremely  high  consequence) 

Supervisory 

Ratings 

(NCOS) 

Task  Delay 

Tolerance 

The  amount  of  delay  that  can  be 
tolerated  between  the  time  the 
need  for  task  performance  becomes 
svldent  and  tha  ttma  that  actual 
performance  begins 

7  -  Point  scale 

(1  •  Extremely  low  [urgency]  to 

7  •  Extremely  high  [urgency]) 

Supervisory 
Ratings  (NCOs) 

Probability  of 

Deficient 

Performance 

The  likelihood  that  the  task  will 
be  performed  poorly 

7 -Point  scale 

(1  •  Never  performed  deficiently  to 

7  -  Very  frequently  performed 
deficiently) 

Supervisory 

Ratings 

(NCOS) 

Immediacy  of 
Performance 

The  time  between  the  task  training 
and  tha  first  task  performance  on 
the  job 

7-  Pokit  scale 

(1  •  Never  performed  to 

7  •  Initially  performed  less  than 

3  months  after  individual  training) 

Supervisory 

Ratings 

(NCOs) 

Relative  Frequency 

The  frequency  of  performing 
the  task 

7-  Point  scale 
(1  -  Very  seldom  to 

7  -  Very  frequently) 

Supervisory 

Ratings 

(NCOS) 

Training  Emphasis 

The  emphasis  that  should  be 
given  to  the  task  during 
systematic  training  (e.g.,  Formal 
school  training.  On -Ih e-Job 
training,  ate.) 

7 -Point  scale 

[1  -  Very  low  emphasis  tc 

7  •  Vary  high  arnphaals) 

Supervisory 

Ratings 

(NCOs) 
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A  recent  study  by  Goldman  (1985)  examined  the  reliability  of 
the  nine  training  factors  in  six  MOSs.  The  study  calculated  the 
average  inter-rater  reliability  of  a  single  rater  (rll]  and  the 
stepped-up  reliability  coefficients  reflecting  the  overall  group 
of  raters  for  a  particular  training  factor  (rkk).  In  general, 
the  study  found  that  the  rll  values  were  moderate  while  the  rkk 
values  were  consistently  high  across  all  MOSs. 

Goldman  also  examined  the  interrelationships  among  the 
training  factors  and  found  statistically  significant  correlations 
across  all  MOSs.  It  was  therefore  not  surprising  that  only  two 
factors  emerged  from  a  factor  analysis  of  the  training  factor 
data  in  each  of  the  six  MOSs. 

Goldman  also  examined  the  success  of  the  seven  NCO-rated 
training  factors  in  predicting  what  tasks  were  actually  trained; 
that  is,  which  AOSP  tasks  were  designated  critical  and  hence 
included  in  the  Soldier's  Manual.  Goldman  conducted  a  stepwise 
discriminant  analysis  to  determine  the  fewest  number  of  training 
factors  needed  to  successfully  identify  the  critical  tasks. 
Goldman  found  that  only  two  factors,  training  emphasis  and 
consequences  of  inadequate  performance,  were  needed  to  predict 
what  tasks  were  designated  critical.  These  results  parallel 
results  from  a  previous  Air  Force  occupational  survey  study  by 
Ruck,  Thompson,  Brown,  &  Stacey  (1978). 

The  results  of  Goldman  &  Ruck,  et  al.  are  encouraging.  They 
suggest  that  only  a  small  number  of  the  training  factors  may  be 
needed  to  predict  training  time,  at  least  for  institutional 
training.  A  small  number  of  factors  would  reduce  the  size  of  the 
TCEA  data  bases,  and  reduce  user  input  requirements. 


Step  12:  System  Allocates  Training  Time  Among  Types 


Output 

The  output  of  this  step  will  be  the  amount  of  time  spent  in 
training  a  task  with  different  "types"  of  training. 

In  initial  institutional  training,  training  type  will  refer 
to  different  types  of  methods  and  media.  To  describe  training 
methods  we  will  use  the  taxonomy  of  training  methods  used  by 
TRADOC  in  its  POIs  (see  Table  11).  To  describe  training  media, 
we  will  use  the  taxonomy  displayed  in  Table  12.  Note  that  this 
taxonomy  primarily  distinguishes  between  different  types  of 
training  devices.  We  propose  this  approach  because  training 
devices  are  the  media  which  are  most  costly  and  have  had  the 
biggest  demonstrated  impact  on  training  task  performance. 
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Table  12.  Example  of  Media  Types* 


Armor 


Conducl-of-Fire  Trainers 

Tank  Trainers  (Driving/Gunnery /Turret) 

Tracked  Vehicle  Driving  Trainer 

Weapons  &  Ordnance 

Firing  Simulator  Systems 


Signal 


Countermeasures  Training  Signal  Transmitting  Set 
Digital  Computer  Trainer 

MaliQQ 

Flight  Simulator 
Composite  Trainer 
Armament  Systems  Trainer 
Hydraulic  &  Electrical  Systems  Trainer 
Flight  Control  System  Trainer 
Panel  Trainers 

Instrument  &  Display  Trainers 
Radar  Operator  &  Target  Simulator 
Power  Plant/Drive  System  Trainers 
Ejection  Seat  Trainer 
Cockpit  Procedures  Trainers 

Air  Defense  Artillery 

Armament  Maintenance  Trainer 
Guided  Missile  Intercept  -  Aerial  Trainer 
Guided  Missile  System  Radar  Signal  Simulator 

Field  Artillery 

Field  Artillery  Trainer 
Programmer  -  Test  Station 


In  unit  training,  training  type  will  refer  to  (a)  training 
level,  either  individual  or  collective,  and  (b)  type  of  training 
media.  The  same  media  categories  used  for  institutional  training 
will  be  used  for  unit  training. 


Input 

External  Input.  None.  This  step  will  be  completely 
automated . 

Internal  Input.  Step  11  will  provide  estimates  of  the  total 
initial  institutional  and  unit  training  time. 


An  on-line  data  base  called  the  Training  Type  Allocation 
Guidelines  will  estimate  the  percentage  of  time  spent  on  each 
type  of  training  in  a  mission  area.  Separate  guidelines  will 
cover  institutional  and  unit  training. 


Process 


The  system  will  apply  the  percentages  from  the  Training  Type 
Allocation  Guidelines  to  the  total  training  time  estimates  from 
Step  11  to  produce  estimates  of  the  time  spent  on  each 
institutional  and  unit  training  type. 


MA&D/DRC  Approach  for  Development 


Development  of  Training  Type  Allocation  Guidelines.  We  will 
develop  the  Training  Type  Allocation  Guidelines  in  a  two-stage 
process.  In  the  first  stage,  we  will  identify  the  percentage  of 
time  spent  on  each  training  type.  For  institutional  training, 
MA&D/DRC  personnel  will  extract  this  information  from  a 
representative  set  of  POIs  for  each  mission  area.  The  POIs 
typically  have  sections  summarizing  the  amount  of  time  spent  with 
different  methods  and  different  types  of  major  media  such  as 
training  devices.  We  will  simply  extract  the  necessary 
information  from  the  POI  and  average  them. 


For  unit  training,  we  will  obtain  information  on  the  time 
spent  on  different  types  of  training  through  the  unit  training 
survey  described  in  Steps  6  and  11  (see  p.  51).  A  section  of 
this  survey  will  ask  unit  training  managers  to  estimate  the 
percentage  of  time  currently  spent  on  different  types  of 
training. 


that 

task 


In  the  second 
each  training 
in  the  future 


stage,  we  will  estimate  the  percentage  of  time 
type  is  expected  to  be  used  in  training  each 
(i.e.,  the  1988-1995  t:me  frame).  We  will 
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present  the  current  percentages  described  above  to  individuals 
from  the  Army  Training  Board  and  Training  Technology  Agency  and 
ask  them  to  modify  them  to  reflect  expected  future  trends  in  Army 
training  technology.  These  agencies  play  a  critical  role  in 
shaping  long-term  training  trends  within  the  Army. 

The  Army  Training  Board's  mission  is  to: 

•  Establish  and  maintain  links  with  the  TRADOC 
integrating  center,  service  schools,  training 
activities,  and  active  and  reserve  component  units. 

•  Foster  communication  and  exchange  of  information 
pertaining  to  development  and  use  of  service  school 
training  materials  and  initiatives. 

•  Collect;  evaluate,  and  disseminate  information  on 
successful  new  training  methods,  management  practices, 
and  materials. 

•  Sponsor  research,  studies,  and  tests  to  improve 
training  development  and  conduct. 

•  Provide  feedback  to  TRADOC  for  the  development  of 
improved  training  materials  and  techniques. 

•  Provide  feedback  to  TRADOC  and  other  Army  activities  in 
the  education  and  training  of  senior  managers 
associated  with  training  and  doctrine  development. 

The  Training  Technology  Agency's  role  is  to  facilitate  the 
transfer  of  new  training  technologies  to  Army  training 
proponents . 


Step  13:  System  Compares  Likely  Training  Time  with 
Maximum  Training  Time 


Output 


During  this  step,  the  system  will  compare  estimated  training 
time  to  the  maximum  training  time  constraint  and  identify 
shortfalls  (that  is,  cases  where  the  estimated  training  time 
exceeds  the  maximum  available  training  time).  Separate  estimates 
will  be  made  for  initial  institutional  and  unit  training. 


Input 


External  Input.  None.  The  user  will  receive  a  report 
describing  any  shortfalls  that  are  identified. 
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Internal  Input.  Step  5  will  provide  estimates  of  maximum 
training  time  for  initial  training.  Step  6  will  provide 
estimates  of  maximum  training  time  for  unit  training.  Step  11 
will  provide  an  estimate  of  the  likely  institutional  and  unit 
training  times  for  each  task. 


Process 

The  system  will  obtain  the  likely  training  time  estimates 
for  each  task  generated  by  Step  11  and  aggregate  them  to  produce 
estimates  of  total  likely  training  time  for  each  HOS.  The  system 
will  develop  separate  estimates  for  institutional  and  unit 
training.  The  institutional  training  estimate  will  be  compared 
with  the  maximum  time  constraint  identified  by  Step  5.  The  unit 
training  estimate  will  be  compared  with  the  maximum  training  time 
constraint  identified  by  Step  6.  A  report  will  be  generated 
listing  any  MOSs  which  have  shortfalls,  the  size  of  the 
shortfall,  and  the  associated  maximum  and  likely  training  time 
estimates . 


MA&D/DRC  Approach  for  Development 

We  will  develop  a  simple  program  to  generate  the  report 
describing  training  time  shortfalls. 


Step  14:  System/User  Iterate  Steps  Until  Shortfalls  Eliminated 
Output 

During  this  step,  the  user  can  change  key  parameters  and 
iterate  the  TCEA  steps  until  training  time  shortfalls  have  been 
eliminated . 


Input 

External  Input.  The  user  must  identify  which  of  the  key 
TCEA  input  parameters  he  or  she  would  like  to  modify  and  input 
new  values  for  these  parameters. 


Process 


The  system  will  present  the  user  with  a  menu  containing  a 
prioritized  list  of  the  input  parameters  that  can  be  changed  to 
eliminate  a  training  time  shortfall.  The  user  will  select  which 
of  these  parameters  will  be  modified.  The  TCEA  will  then  guide 
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The  user  through  the  steps  to  enter  new  values  for  these 
parameters  and  calculate  the  impact  on  training  time 
shortfalls.  Table  13  lists  the  key  parameters  which  can  be 
modified,  the  steps  where  the  modifications  are  made,  and  the 
steps  which  must  be  iterated  to  calculate  the  impact  on  training 
time  shortfalls.  The  system  will  prompt  the  user  to  provide  a 
description  of  the  rationale  underlying  any  changes  to  the  key 
input  parameters. 


MA&D/DRC  Approach  for  Development 

We  will  create  software  to  provide  the  branching  from  step- 
to-step  that  will  be  associated  with  TCEA  iteration.  We  will 
create  additional  software  to  help  the  TCEA  keep  track  of 
parameter  values  associated  with  any  particular  iteration  or  set 
of  iterations. 
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Table  13.  Recommended  Sequence  for  Modifying  TCEA  Parameters. 


E3-75 


Unit  Training  Time  Step  6  Steps  1 3  and  1 4 

Constraint  Factors 


DESCRIPTION  AND  DEVELOPMENT  OF  AUTOMATED  COMPONENTS 

We  must  develop  four  different  types  of  software  to  support 
the  TCEA : 


•  Input/Output  frames  that  provide  mechanisms  for  users 
to  select  options  or  commands,  input  data,  or  generate 
output  reports. 

•  Models  that  describe  quantitative  algorithms  for 
transforming  the  data. 

•  Permanent  data  base  libraries  for  critical  data. 

•  Temporary  files  that  store  data  associated  with  a 
particular  user's  TCEA  application. 


Details  on  the  specific  TCEA  data  bases  and  models  appear 
below . 

An  Applications  Manager  will  control  all  of  the  software 
components . 

We  are  purposely  repeating  material  in  Sections  4  and  5.  We 
hope  doing  so  will  make  it  easier  to  read  each  section 
independently . 

Data  Base  Libraries 


Table  14 
developed  for 
describes  the 
each  of  these 


lists  the  data  base  libraries  that  must  be 
each  step  of  the  TCEA.  The  section  which  rollows 
data  sources  and  procedures  we  will  use  to  develop 
libraries . 


Development  of  New  System  Fielding  Schedule.  The  New  System 
Fielding  Schedule  will  describe  the  ma^or  new  weapon  systems 
which  are  expected  to  be  fielded  in  each  mission  area,  the 
systems  they  will  replace,  and  the  date  they  are  expected  to  be 
fielded.  All  of  the  information  needed  to  produce  the  New  System 
Fielding  Schedule  is  available  in  the  AMIM.  We  will  simply 
review  these  data  and  place  them  in  the  data  base. 


Development  of  the  Unit  Training  Time  Constraint  Data 
Base .  The  Unit  Training  Time  Constraint  data  base  will  list  the 
maximum  amount  of  time  that  different  types  of  units  are  expected 
to  have  to  devote  to  weapon  system-specific  training.  The 
estimates  will  be  broken  out  by  mission  area  and  by  unit 
location . 

Unfortunately,  the  information  needed  to  complete  the  Unit 
Training  Time  Constraint  data  base  is  not  available  in  any 
automated  data  base.  To  develop  this  information,  we  propose  to 
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Table  14.  List  of  Data  Bases  in  the  TCEA. 


New  System  Fielding  Schedule 

Unit  Training  Time  Constraint  Data  Base 

Major  System  by  Unit  Data  Base 

Equipment  by  Major  System  Data  Base 

Training  Factor  Data  Base 

Task  Verb  Thesaurus 

Task  Taxonomy 

Equipment  Taxonomy 

Training  Type  Allocation  Guidelines 
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send  out  a  survey  questionnaire  to  a  representative  sample  of 
unit  training  managers.  The  survey  will  provide  data  for  several 
of  the  TCEA  steps.  Table  15  lists  the  type  of  questions  that 
will  be  included  in  the  survey  and  the  relevant  TCEA  steps. 

We  will  develop  the  survey  in  accordance  with  state-of-the- 
art  guidelines  for  mail  surveys  (Altschuld  &  Lower,  1984; 

Dillman,  1978;  Dyer,  et  al.,  1976;  Eudos,  1983;  Sudman  7 
Bradburn,  1982;  Lockhart,  1984).  DRC  has  successfully  conducted 
a  number  of  surveys  for  ARI.  These  surveys  include  studies  on 
the  Joint  Optical  Information  Network  (JOIN)  (Lockhart  et  al., 
1987),  Analysis  of  the  Soldier  In  Europe  Survey  Data  (Lockhart, 
1987),  and  the  user  survey  for  the  Man  Integrated  Systems 
Technology  (MIST)  (Dynamics  Research  Corporation,  1965). 

We  will  identify  the  type  of  units  that  operate  or  maintain 
major  weapon  systems  and  select  a  stratified  sample  of  battalion 
unit  training  managers  from  each  major  command. 

To  obtain  a  high  response  rate,  we  will  mail  pre-notices, 
questionnaires,  and  follow-up  items  to  the  participants.  In  a 
recent  DRC  survey  (Lockhart,  et  al.,  1987),  we  used  a  series  of 
five  mailings  to  increase  the  number  of  respondents.  Table  16 
outlines  this  mailing  schedule. 

The  pre-notice  informs  the  participants  that  the  question¬ 
naire  is  coming  and  asks  them  for  their  assistance.  The  survey 
is  mailed  out  the  week  after  the  pre-notice.  A  reminder  asks  all 
participants  to  complete  the  survey  and  provides  a  toll-free 
number  tney  can  use  for  help.  The  second  questionnaire  is  sent 
to  soldiers  who  have  not  returned  the  first  questionnaire  within 
four  weeks.  A  thank  you  letter  is  sent  to  each  participant  after 
receipt  of  a  completed  questionnaire. 

Development  of  the  Major  System  by  Unit  Data  Base.  The 
Major  System  by  Unit  Data  Base  will  list  the  major  systems 
associated  with  different  unit  types.  A  list  of  the  major 
systems  associated  with  each  unit  type  can  be  obtained  from  the 
TOE  associated  with  that  unit. 

Development  of  the  Equipment  by  Major  System  Data  Base. 

This  data  base  will  list  the  major  equipment  items  associated 
with  major  weapon  systems  in  each  mission  area.  We  expect  about 
10  to  15  of  these  items  per  weapon  system.  Lists  of  the  major 
equipment  items  associated  with  a  weapon  system  can  be  found  in 
several  sources  including  the  AMIM  and  the  AMMDB .  (DRC  has  on¬ 
line  access  to  the  AMMDB). 
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Table  15.  Type  of  Questions  to  Be  included  In  Unit  Training  Time  Survey. 
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These  are  not  the  actual  questions  that  will  be  Included  In  the  survey.  They  are  the  topics  that  the  questions  will  cover. 


Tablet  6.  Proposed  Mailing  Schedule  for  Unit  Training 

Time  Survey. 


Mailing 

Pre-notice 

Survey 

Reminder 

Second  Questionnaire 
Thank  You 


Date 

One  Week  Prior  to  Survey 

One  Week  After  Survey 
Four  Weeks  After  Survey 
After  Return  of  Survey 
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Data  for  Training  Factor  Data  Base.  We  will  obtain  all  of 
the  data  in  the  Training  Factor  Rating  Data  Base  from  the  CODAP 
data  base,  which  is  part  of  the  Army  Occupational  Survey  Program 
(AOSP).  Table  17  describes  the  nine  training  factors  in  the 
CODAP  data  base.  During  the  development  of  the  TCEA,  we  will 
request  a  tape  extract  that  contains  results  from  all  of  the 
training  factor  surveys  conducted  by  AOSP  since  1981,  the  year 
when  these  surveys  were  initiated.  We  will  load  into  the  TCEA 
Training  Factor  Data  Base  only  those  factors  which  are  part  of 
our  training  time  prediction  model;  that  is,  those  factors  that 
are  significant  predictors  of  institutional  or  unit  training 
time.  See  the  section  entitled  "MA&D/DRC  Approach  for 
Development,"  pp.  64-66  for  detailed  description  of  our  approach 
to  developing  this  model. 

Our  approach  does  not  require  that  the  Training  Factor  Data 
Base  contain  task  factor  ratings  for  all  Army  tasks.  All  that  we 
need  are  ratings  for  a  fairly  large  number  of  tasks.  The  purpose 
of  the  Training  Factor  Data  Base  is  to  provide  the  user  with  some 
baseline  data  from  a  similar  task.  If  data  are  not  there  for  one 
particular  task,  the  user  can  identify  another  similar  task  which 
does  have  existing  data. 

Development  of  Task  Verb  Thesaurus.  There  are  several 
microcomputer-based  data  base  management  systems  which  already 
have  front-end  natural  language  processors.  If  we  use  one  of 
these  data  bases  to  store  the  Task  Factor  Ratings  Data  Base,  we 
should  be  able  to  use  its  processor  to  identify  verb  synonyms. 

If  we  don't  use  one  of  these  data  bases,  we  can  create  a  simple 
thesaurus  of  approximately  100  verbs. 

Development  of  Task  Taxonomy.  The  task  taxonomy  will  group 
together  task  actions  based  on  similarities  in  their  training 
factor  ratings.  To  create  this  taxonomy,  mean  scores  on  each 
training  factor  measure  will  be  created  for  each  task  action  by 
averaging  across  all  task  which  use  that  action  in  the  data 
base.  Profile  difference  scores  will  provide  a  measure  of  the 
similarity  of  one  action  to  another.  The  profile  difference 
scores  will  be  entered  into  a  cluster  analysis  routine,  which 
will  produce  the  task  action  clusters  to  form  the  taxonomy. 

Table  18  summarizes  some  of  the  different  clustering  techniques 
that  could  be  used  in  this  analysis.  We  will  select  one  of  these 
techniques  during  development  of  the  detailed  design 
specifications . 

Development  of  Equipment  Taxonomy.  The  equipment  taxonomy 
will  group  together  equipment  items  based  on  similarities  in  the 
training  factor  ratings  of  the  tasks  associated  with  these 
equipment  items.  To  create  this  taxonomy,  mean  scores  on  each 
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Table  17.  Training  Factor  Rating  Scales. 


Training  Factor 

Definition 

Scale 

Source  of 
Rating 

Percent  of  member  s 
performing 

Percent  of  fob  Incumbents 
who  perform  the  task 

Percent  scale  based  upon 
proportion  of  Yaatfo 

responses 

Self-Report 

(Enlisted) 

Average  percent  of 
time  spent  by  members 
performing 

Average  percent  time  spent 
on  the  task  rslatfvs  to  time 
spent  on  all  other  teaks 

7  -  Point  Relevant  Time 

Spent  scale 

(1  -  Very  much  below  average  to 

7  -  Vary  much  above  average) 

Self-Report 

(Enlisted) 

Task  Learning 

Difficulty 

The  amount  of  Ume  required 
to  learn  to  perform  the  teak 
satisfactorily 

7 -Point  scale 
(1  •  Extremely  low  teaming 
difficulty  to 

7  •  Extremely  high  learning 
difficulty) 

Supervisory 

Ratings 

(NCOS) 

Consequences  of 
Inadequate 

Pt1orm*nc* 

The  consequences  of  performing 
the  task  Inadequatafy  upon 
personnel  injury,  teas  of  life, 
equipment  demege 

7*  Point  scale 

(1  -  Extremely  low  consequence  to 

7  •  Extremely  high  consequence) 

Si  rrvisory 

Ra.tngs 

(NCOS) 

Task  Delay 

Tolerance 

The  amount  of  daisy  that  can  be 
tolerated  between  the  time  the 
need  for  task  performance  becomes 
evident  and  the  time  that  actual 
performance  begins 

7 -Point  scale 

(1  -  Extremely  low  [urgency]  to 

7  •  Extremely  high  [urgency]) 

Supervisory 
Ratings  (NCOs) 

Probability  of 

Deficient 

Performance 

The  likelihood  that  the  task  will 
be  performed  poorly 

7  •  Point  scale 

(1  •  Never  performed  deficiently  to 

7  •  Very  frequently  performed 
deficiently) 

Supervisory 

Ratings 

(NCOs) 

Immediacy  of 
Performance 

The  time  between  the  task  training 
and  the  first  task  performance  on 
the  job 

7-  Point  scale 

(1  •  Never  performed  to 

7  •  foitlally  performed  less  than 

3  months  after  Individual  training) 

Supervisory 

Ratings 

(NCOs) 

Relative  Frequency 

The  frequency  of  performing 
the  task 

7-  Point  scale 
(1  -  Vary  seldom  to 

7  •  Vary  frequently) 

Supervisory 

Ratings 

(NCOs) 

Tralnkig  Emphasis 

The  emphasis  that  should  be 
given  to  the  task  during 
systematic  training  (e.g.,  Formal 
school  training,  On-the-Job 
training,  etc.) 

7-  Point  scale 

(1  •  Very  low  emphasis  to 

7  •  Vtry  high  emphasis) 

Supervisory 

Ratings 

(NCOs) 
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Table  18.  Potential  Clustering  Techniques 


1 

j  TECHNIQUE 

CLASSIFICATION 

SELECTION  CRITERION 

COMPUTATIONAL  1 
COMPLEXITY 

COMMENTS 

Single  Linkage 

Hierarchical 
-  Linkage 

Merge  clutter  A  end 
putter  S  for  which 

SA8  *  mm  |S*il**A.  l,Bl 

■  the  tmeilart- 

low 

e  tiieellent  intuitive  j 
darrty 

Comotet*  Unugi 

Hierarchical 
—  Linauapa 

Merge  duttar  A  and 
d utter  B  for  which 

sAb  *  "**  {Sij|t«A.  kb| 

It  the  nnalleet. 

low 

e  excellent  mturtne 
darrty 

Centroid 

Hierarchical 

Merge  duttar  A  and 
cfuttar  S  for  erhich 

t  pittance  between  the 
SaB  “  \  centroid  of  A  and 
|  centroid  of  B 
it  the  tmallect. 

fliodwti 

e  automated 

a  excellent  mtu.tr re 
clarity 

The  Mart)  Crrtenon 

Hierarchical 
—  Error  Sum  of 
Square* 

Merge  the  duttar*  to  that 
total  wrthin  group  error  arm 
of  muara*  a  minimi* eo. 

high 

e  automated 

e  heavily  influenced 
by  group  *ue 

e  moderate  imurtrva 
darrty 

Minimum  Total  Within  Group 
Sum  of  Sou  arm  m  the  New 
Clutter 

Hierarchical 
—  Error  Sum  of 
Square* 

Merge  the  duttar*  to  that 
the  wrthirvgroup  error  aim 
of  *ouarae  m  the  new  cfuttar 
a  tre rumored. 

high 

(drghtty  lee*  then 
die  Wen  Crrtarion) 

e  automated 
e  heavily  mfluencad 
by  group  toe 

a  moderate  mturtne 
darrty 

Minimum  A  erege  Within 

1  Group  Sum  of  Sou  arm  m  the 
Mam  Clutter 

Hierarchical 
—  Error  Sum  of 
Square* 

Marge  the  duttar*  to  that 
the  arereoe  wtthirvgroup 
error  aim  of  aquaraa  in  the 
new  duttar  it  rmnuntaad. 

i 

high 

a  automated 

a  heavily  influenced 
by  group  un 

a  moderate  mturtne 
darrty 

|  Forgy  «  AlgorTthm 

NooJimrwrchtcal 
-  Soad  potrro 

1 

'  A «rgn  each  cadt  to  the  Had 
tatk  to  which  rt  it  tnott 
umiiar  Clutter  certtrotdi 
become  new  lead  point*. 
Iterate. 

moderate  to 
high 

e  moderate  intuitive 
denty 

Jartcw  i  Algocrtfim 

: 

\ 

i 

1  Non-hierarchical 
|  -  Seed  point* 

1 

Aaign  each  ta*  to  the  teed 
teak  to  which  it  it  mort 
limiter.  New  teed  point* 
we  reflection  of  old  teed 
point*  through  cemroidi. 
Iterate. 

high 

•  imuitiv«4y  hart  to 

MacOuaan't 

1  K-maana  algorithm 

Norv-hwrerchml 
-  Seed  point* 

Aeegn  one  tadr  to  the  lead 
to  which  rt  it  non  timrler. 

C kittar  cemroidi  become  now 
mad  point*.  Iterate. 

modnn  *o 

e  moderate  intuitive 
dariry 

Jorgenaon  t  TEEM  Algorrthm 

Non-hwrarehic*l 
—  Error  Sum  of 

Square* 

Merge  duttar  A  end  B  for 
which  a  linear  eombmenon 
of  total  between  group  error 
turn  of  m»uera*  mmu i  total 
between  group  error  aim  of 
atuara*  a  majtimtcad. 

high 

e  automated 

e  heavily  influenced 
by  number  of 
group* 

e  moderate  intuitive 
demy 

Port -Clutter  Allocation 

Depend  am  on 

After  tome  form  of 

deoendem  of 
duttanng  tachmaue 
m  it  iefty  uetd 

e  automated  vemoni 

A>9onttifm 

duttarmg 
teehniqu* 
mroally  uaad 

d vi tier  mg  ha*  bean  performed, 
aoign  mch  tadt  to  ell 
duttar*  for  which  ft  mddt*  Of 
eaedadt  the  appropnete 
nmilarrry  meet  ire  met  by 
other  duttar  memper* 

e  technique  to  include 
or*i  in  more  then 

1  duttar 

•  moderate  intuitive 
derny 

1  L  tnk  »9<  DiwC  network 

N  ofvh  Hprw  ch  id  < 

Link  together  t»i  for  which 
the  mtertufc  dittanca  n  let* 
than  or  equal  to  tome 
meaimim,  D. 

vnodorat* 

! 

e  excellent  intuitive 
denrv 

e  allow*  task  to  be 
j  miiqned  to  more 

men  one  clutter 
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training  factor  measure  will  be  created  for  each  equipment  by 
averaging  across  all  the  tasks  associated  with  that  equipment 
item  in  the  data  base.  Profile  difference  scores  between  the 
equipment  items  on  the  mean  training  factor  ratings  will  then  be 
created.  These  profile  difference  scores  will  provide  a  measure 
of  the  similarity  of  one  equipment  item  to  another.  The  profile 
difference  scores  will  be  entered  into  a  cluster  analysis 
routine,  which  will  produce  the  equipment  item  groups  to  form  the 
taxonomy . 

Development  of  Training  Type  Allocation  Guidelines.  We  will 
develop  the  Training  Type  Allocation  Guidelines  in  a  two-stage 
process.  In  the  first  stage,  we  will  identify  the  percentage  of 
time  typically  spent  in  training  with  each  training  type.  For 
institutional  training,  MA&D/DRC  personnel  will  extract  this 
information  from  a  representative  set  of  POIs  for  each  mission 
area.  The  POIs  typically  have  sections  summarizing  th2  amount  of 
time  spent  with  different  methods  and  different  types  of  major 
media,  such  as  training  devices.  We  will  simply  extract  the 
necessary  information  from  the  POIs  and  average  them. 

For  unit  training,  we  will  obtain  information  on  the  time 
taken  up  by  different  types  of  training  through  the  unit  training 
survey  described  in  Steps  6  and  11  (see  information  on  survey  p. 
51).  A  section  of  this  survey  will  ask  unit  training  managers  to 
estimate  the  percentage  of  time  currently  spent  on  different 
types  of  training. 

In  the  second  stage,  we  will  estimate  the  percentage  of  time 
that  each  training  type  will  use  in  training  each  task  in  the 
future  (i.e.,  the  1988-1995  time  frame).  We  will  present  the 
current  percentages  described  above  to  individuals  from  the  Army 
Training  Board  and  Training  Technology  Agency  and  ask  them  to 
modify  them  to  reflect  expected  future  trends  in  Army  training 
technology.  These  agencies  play  a  critical  role  in  shaping  long¬ 
term  training  trends  within  the  Army. 


Models 

Development  of  Training  Time  Prediction  Model.  To  develop 
the  training  time  prediction  equations,  we  will  conduct  a  series 
of  regression  analyses  using  the  training  factors  as  predictors 
and  the  time  to  train  a  task  as  a  criterion.  We  will  develop 
separate  equations  to  predict  institutional  and  unit  training 
time  within  each  mission  area. 

To  develop  the  data  for  th^  regression  analyses,  we  will 
select  a  sample  of  tasks  for  each  mission  area  from  the  CODAP 
data  base.  We  will  extract  data  on  the  nine  training  factors 
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from  this  data  base.  We  will  obtain  data  on  the  time  spent  in 
training  the  tasks  in  initial  training  by  examining  the  Program 
of  Instruction  for  the  initial  institutional  training  course. 

(This  course  can  be  identified  by  examining  the  Trainer's  Guide 
for  the  MOS  associated  with  the  task.) 

We  will  estimate  the  annual  amount  of  time  spent  in  unit 
training  from  a  survey  questionnaire  sent  to  unit  training 
managers.  We  will  use  this  same  survey  to  determine  the  maximum 
time  available  to  train  all  system  tasks.  Page  51  provides  a 
more  detailed  description  of  the  procedures  employed  in  this 
survey . 

Once  we  have  collected  data  on  training  time,  we  will 
conduct  a  series  of  step-wise  regression  analyses  using  the 
institutional  training  time  measures  as  criterion  variables.  In 
conducting  the  regression  analyses,  we  will  attempt  to  identify 
the  minimum  number  of  training  factor  variables  needed  to  predict 
training  time  for  each  mission  area.  By  minimizing  the  number  of 
training  factor  variables,  we  can  reduce  the  number  of  ratings 
that  the  users  must  provide. 

Application  of  the  Training  Time  Prediction  Equations .  The 
TCEA  will  assist  the  user  in  inputting  data  on  the  training 
factors  included  in  the  training  prediction  model.  Once  these 
data  are  entered,  the  TCEA  will  apply  the  regression  equations  to 
predict  institutional  and  unit  training  time. 

The  TCEA  will  help  the  user  identify  "similar1'  tasks  which 
a1 ready  have  training  factor  ratings.  The  TCEA  will  extract 
these  ratings  from  CODAP  and  store  them  in  an  on-line  data  base 
called  the  Training  Factor  Rating  Data  Base.  The  system  will 
automatically  identify  a  "similar"  task  by  matching  task  action 
and  noun  statements  with  the  categories  in  Task  and  Equipment 
Taxonomies.  The  user  can  review  or  modify  this  assignment,  or 
select  another  task  from  the  Training  Factor  Rating  Data  Base, 
the  user  will  be  able  to  review  the  ^raining  factor  ratings 
associated  with  the  similar  tasks  3  update  them  to  reflect  the 
new  system's  equipment  design  and  system  performance 
requ i r ements . 

Develc  ment  of  ATRRS  Data  Extraction  Program.  The  program 
for  extracting  information  from  ATRRS  will  present  no  major 
technical  problems.  DRC  already  has  the  ATRRS  user's  manuals  and 
the  ATRRS  data  element  description.  The  specific  data  elements 
from  which  we  will  extract  data  are  MOS  (data  element  E-YY), 
course  number  (CRS-NO),  course  title  (CRS-TITLE),  course  annual 
student/trainee  input  capacity  (ANN-CAP),  and  course  length  in 
days  (CRS-DAYS).  The  course  number  code  identifies  which  courses 
are  initial  training  courses. 
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User  Interface 


As  noted  in  the  Product  Requirements  section,  the  TCEA  must 
be  designed  for  users  with  little  or  no  computu-r  background. 

Users  must  be  able  to  implement  the  TCEA  through  easy-to-lear n 
and  use  human-computer  dialogue  techniques.  A  number  of 
researchers  (e.g.,  Williges,  1986;  Parrish,  Gates,  &  Munger, 

1981;  Shaw  and  McCauley,  1985;  and  Smith  &  Mosier,  1984)  have 
developed  guidelines  for  developing  such  techniques.  These 
guidelines  are  perhaps  best  summarized  in  Williges'  general 
principles  for  user  interface  development  (1986).  They  are: 

•  Compatibility  Principle  -  Minimize  the  amount  of 
information  recoding  that  will  be  necessary. 

•  Consistency  Principle  -  Minimize  the  difference  in 
dialogue  both  within  and  across  various  human-computer 
interfaces . 

•  Memory  Principle  -  Minimize  the  amount  of  information 
that  the  analyst  must  maintain  in  short-term  memory. 

•  Structure  Principle  -  Assist  the  analyst  in  developing 
a  conceptual  representation  of  the  system  structure  so 
that  he  or  she  can  navigate  through  the  interface. 

•  Feedback  Principle  -  Provide  the  analyst  with  feedback 
and  error  correction  capabilities. 

•  Workload  Principle  -  Keep  the  analyst's  mental  workload 
within  acceptable  limits. 

During  the  development  of  detailed  design  specifications,  we 
will  generate  user  interface  guidelines  for  the  TCEA  from  a 
subset  of  existing  guidelines  in  Williges  (1986),  Smith  &  Mosier 
(1984),  etc.  The  guidelines  will  shape  the  development  of  new 
software  and  the  selection  of  off-the-shelf  software  for  the 
TCEA.  To  maintain  a  consistent  user  interface,  we  propose  that 
the  same  guidelines  be  applied  to  all  six  (MPT)2  products. 


Hardware  and  Software  Configuration 

Final  selection  of  a  hardware  and  software  configuration 
cannot  occur  until  detailed  design  specifications  are  developed. 
More  information  is  needed  about  users  and  the  type  of  hardware 
and  software  available  to  them  when  the  TCEA  is  implemented .  It 
is  also  vital  to  know  the  (MPT)2  concepts  will  be  implemented. 
Ideally,  all  six  (MPT)2  products  should  use  the  same  basic 
hardware  and  software.  But  consistency  is  particularly  important 
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for  (MPT) ^  products  witn  the  same  primary  users.  The  sections 
below  outline  our  current  view  of  the  (MPT)2  hardware  and 
software  configurations.  They  are  based  on  an  integrated 
assessment  of  MA&D/DRC's  MPT  concepts  or  the  six  (MPT)2  products. 


Potential  Hardware  Configuration 

Our  current  concept  of  the  (MPT)2  computer  configuration 
includes  an  MS-DOS-compatible  microcomputer  with  at  least  20 
megabytes  of  hard  disk  storage,  at  least  two  megabytes  of  RAM, 
two  floppy  disk  drives,  a  modem,  and  a  132-column  printer.  A 
large  amount  of  external  storage  will  be  needed  to  store  the 
large  data  bases  associated  with  the  (MPT)2  products.  A  large 
amount  of  RAM  is  also  needed  to  decrease  response  time  and  allow 
the  simulation  that  is  expected  to  occur  in  Products  1,  5,  and  6. 
Ideally,  the  microcomputer  would  be  based  on  at  least  an  80286 
processor  (i.e.,  an  AT-class  processor)  and  include  an  advanced 
color  monitor.  Depending  on  the  mechanism  chosen  for  menu 
selection,  a  mouse  or  a  lightpen  may  also  be  part  of  the 
configuration . 


Potential  Software  Configuration 

By  the  time  the  (MPT)2  products  are  coded,  we  expect  MS-DOS 
version  5.0  to  be  available.  To  facilitate  integration  across 
products,  all  new  software  created  for  (MPT)2  should  be  written 
in  a  single,  highly  structured  language  such  as  C.  C,  like  Ada, 
has  proven  to  be  extremely  machine-independent.  MA&D  has  used  C 
to  implement  Micro  SAINT. 

Off-the-Shelf  Versus  New  Software.  Several  common  software 
functions  are  expected  to  extend  across  most,  if  not  all,  the 
(MPT)2  products.  For  example,  all  six  aids  will  need  a  data  base 
management  system  for  storing  and  accessing  data.  Several 
products  may  also  need  spreadsheet  software  to  perform  simple 
calculations.  Others,  like  the  TCEA,  may  need  communications 
software  to  access  data  bases  that  reside  on  other  computers. 
Still  other  aids  such  as  Products  1,  5,  and  6  m?»v  need  simulation 
software  such  as  Micro  SAINT  to  represent  complex  system-human 
interactions . 

We  can  use  two  basic  approaches  to  develop  the  (MPT)2  soft¬ 
ware.  We  can  either  use  off-the-shelf  software  product  or 
develop  new  software.  The  advantage  of  off-the-shelf  software  is 
that  it  can  drastically  reduce  software  development  and  documen¬ 
tation  costs.  Although  the  cost  savings  would  not  significantly 
impact  the  next  phase  of  (MPT)2  (development  of  detailed  design 
specifications),  they  could  effect  Phase  3  software  development 
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greatly.  Another  advantage  of  off-the-shelf  software  packages  is 
that  they  often  include  additional  features  or  add-ons  that  go 
beyond  (MPT)2  minimum  essential  requirements  at  little  or  no 
additional  cost.  In  addition,  cf f-the-shel f  software  can  be  used 
for  applications  other  than  (MPT)2. 

The  major  disadvantage  of  off-the-shelf  software  is,  of 
course,  that  software  that  can  meet  (MPT)2  functional  require¬ 
ments  might  not  exist.  Another  disadvantage  is  that  it  is  often 
impossible  to  modify  the  user  interface  in  off-the-shelf  pack¬ 
ages.  Typically,  one  has  to  "take  or  leave"  the  user  interface. 
If  more  than  one  off-the-shelf  package  is  used,  one  runs  the  risk 
of  having  inconsistent  interfaces.  Another  disadvantage  of  off- 
the-shelf  software  for  a  Government  system  such  as  (MPT)2  is  the 
proprietary  nature  of  this  software.  In  order  to  distribute  the 
(MPT)2  products  successfully,  users  must  be  able  to  acquire  the 
software  associated  with  them  at  little  or  no  additional  cost. 
Many  software  companies  will  not  grant  the  Government  distribu¬ 
tion  rights  to  their  software  or  will  charge  each  Government 
agency  using  the  product  the  full  price. 

Another  related  problem  is  that  many  Army  organizations 
outside  the  Rs-D  community  have  difficulty  purchasing  software 
that  is  not  on  the  GSA  list.  Fortunately,  many  useful  software 
products  are  being  added  to  the  GSA  list.  One  such  product  is 
ENABLE’".  ENABLE*  is  an  integrated  software  package  with 
spreadsheet,  graphics,  data  base  management,  telecommunications, 
and  word  processing  modules.  The  ENABLE*  data  base  management 
system  can  store  up  to  65,000  records  per  file.  Its  spreadsheet 
can  utilize  up  to  65,025  cells.  Its  graphics  package  produces 
high  resolution  black  and  white  or  medium  resolution  color 
graphics  and  can  create  graphics  from  the  DBMS  or  spreadsheet 
module.  ENABLE*  also  has  a  Master  Control  module  that  provides  a 
window  to  DOS  and  a  built-in  menu  generation  capability  for  up  to 
eight  windows.  These  windows  can  be  sized,  shaped,  overlapped, 
or  zoomed  to  full  screen.  ENABLE™  can  be  purchased  from  the  GSA 
list  for  approximately  $400. 

As  we  develop  detailed  design  specifications  for  (MPT)2,  we 
will  select  an  approach  (new  software  versus  specific  off-the- 
shelf  software)  to  implement  the  software  components  of  each 
(MPT)2  product.  This  approach  must  be  congruent  with  the 
approach  taken  to  develop  other  (MPT)2  products. 
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TECHNOLOGY  TRANSFER  ISSUES 


Training  Strategy 

Our  goal  is  to  design  a  set  of  automated  (MPT)^  tools  that 
the  user  can  implement  immediately  without  external  training.  To 
accomplish  this  goal,  we  will  (1)  employ  a  user  interface  that 
requires  very  little  computer  experience  (see  "Description  and 
Development  of  Automated  Components,"  p.  74),  (2)  provide  on-line 
help  to  explain  alternatives,  and  (3)  automate  as  many  analytical 
and  data  collection  activities  as  possible. 

The  only  external  training  we  believe  may  be  necessary  is  a 
brief  pamphlet.  This  pamphlet  will  describe  the  hardware  and 
software  needed  to  run  the  aid,  how  to  load  the  aid  onto  the 
computer,  and  what  input  data  (;f  any)  users  should  have  on  hand 
before  to  use  the  aid.  We  also  recommend  developing  a  small 
manual  on  how  to  use  the  results  of  the  aid  in  the  acquisition 
process.  The  experienced  MPT  user  will  not  need  this  training. 
Frequently,  however,  users  with  no  background  in  the  acquisition 
process  or  MPT  must  use  the  aid.  Such  users  will  need  an 
overview  of  the  acquisition  process  and  how  the  aid  can  help  them 
during  the  process.  These  inexperienced  users  will  also  need 
examples  of  product  input  and  output. 


Means  for  Achieving  Institutionalization 

During  the  development  of  design  specifications  for  this 
product  (Option  1),  we  will  produce  a  detailed  fielding  plan  for 
the  product.  This  fielding  plan  will  describe  the  distribution 
of  the  aid's  methods,  hardware,  software,  documentation,  and 
training  programs  (media,  instructors,  etc.)  to  specify  Army 
users  in  specific  Army  organizations.  The  plan  will  be  analogous 
to  the  Materiel  Fielding  Plan  for  Army  weapon  systems.  We  will 
develop  a  draft  of  this  plan  during  Option  1  and  a  final  version 
during  Option  2. 

At  the  present  time,  we  believe  that  successful  implemen¬ 
tation  will,  at  a  minimum,  ’-equire  the  following  activities: 

Identification  of  Specific  Users.  We  must  identify  specific 
users  of  each  product  and  describe  the  specific  MAP  activities 
and  documents  the  aid  will  feed.  This  will  ensure  that  the 
product  has  a  "real  world"  use.  The  section  entitled  "Product 
Requirements"  describes  our  approach  for  accomplishing  this. 

Incorporation  of  Users  in  Product  Development.  To  ensure 
that  the  product  meets  their  needs,  we  will  include  users  in  the 
product  development  process.  At  a  minimum,  they  should  use  the 


product  during  the  external  demonstration  that  will  take  place 
during  Option  2.  Ideally,  they  should  also  review  the  final 
concept  papers  and  the  detailed  design  specifications  from  Option 
1.  We  stand  ready  to  help  ARI  coordinate  user  participation  in 
product  development. 

Incorporation  of  Acceptability/Usability  Requirements  into 
Product  Specifications.  We  have  incorporated  acceptability  and 
usability  requirements  into  the  requirements  specification  for 
each  aid  (see  "Acceptability/Usability  Requirements",  p.  15). 
These  requirements  include  features  that  will  make  the  product 
easy  to  use  (e.g.,  clear  documentation,  on-line  help.  etc.). 
During  design  specification  (Option  1),  we  will  develop  detailed 
user  interface  guidelines.  To  ensure  a  consistent  interface,  the 
same  guidelines  will  be  applied  to  every  product. 

Instruction  of  Key  Personnel.  We  propose  that  key  personnel 
receive  detailed  training  at  ARI  headquarters  immediately  after 
ARI  has  accepted  the  aid.  These  key  personnel  will  be 
individuals  likely  to  become  (1)  experts  in  using  the  aid,  (2) 
instructors  in  using  the  aid,  and  (3)  consultants  for  ongoing 
applications  of  the  aid.  At  the  ~>resent  time,  we  recommend  that 
these  key  personnel  be  staff  members  from  ARI's  System's  Research 
Lab,  MANPRINT  support  personnel  in  ARI  field  offices,  and 
MANPRINT  Policy  Office  staff  within  DCSPER. 

Demonstrate  Aid  at  User  Sites.  We  also  recommend  that  the 
aid  be  demonstrated  at  all  primary  user  sites  by  contractor 
personnel  or  the  Key  personnel  who  were  trained  at  ARI  head¬ 
quarters.  These  demonstrations  would  provide  hands-on  training 
in  the  aid  software  using  "real  world"  examples,  describe  the 
benefits  of  the  product,  and  show  users  how  the  product  can  help 
them  produce  MAP  products. 

Software  Maintenance.  Specific  Army  organizations  must  be 
identified  that  can  continuously  update  software,  documentation, 
and  training  to  reflect  user  applications  and  evolving  needs. 

On-line  Help.  We  recommend  that  a  "hotline"  be  established 
to  answer  users'  questions  and  document  problems.  The  hotline 
would  be  like  those  used  by  commercial  software  vendors  and  the 
MANPRINT  Policy  Office, 

Incorporation  into  Army  Training  Programs  and  Regulations. 
Army  training  courses  for  MANPRINT,  project  management,  etc., 
must  be  modified  to  describe  how  the  aid  can  help  users  during 
the  MAP.  Regulations  and  pamphlets  in  these  areas  must  be 
modified  in  the  same  way. 
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We  estimate  that  it  will  take  approximately  80  hours  to 
apply  the  TCEA. 
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APPENDIX  A — SCRIPTS  FOR  AUTOMATED  STEPS 


SOFTWARE  DEVELOPMENT  PLAN 


The  MPT  Decision  Support  System  (DSS)  will  be  developed 
using  the  time-proven  system  process  that  all  successful  software 
organizations  employ.  This  process  is  outlined  in  MIL-STD-2167/ 
2168  and  DoD  Standard  7935.  To  further  ensure  success,  we  have 
adapted  the  process  described  in-  these  standards  to  meet  this 
project's  unique  needs.  The  process,  illustrate-  at  Figure  A-l, 
consists  of  three  steps  and  the  products  resulting  from  them. 

Step  1  -  Requirements  Analysis 

The  requirements  analysis  identifies  the  specific  functions 
that  the  system  must  perform.  High  level  functional  requirements 
were  identified  in  the  concept  papers.  Detailed  functional 
requirements  will  be  developed  in  Phase  2,  Detailed  Design 
Specifications.  The  requirements  will  describe  the  context, 
constraints,  and  functional  requirements.  Context  requirements 
include : 


•  The  general  requirements  that  the  new  system  intends  to 
meet ; 

•  The  environment  in  which  the  new  system  will  exist; 

•  How  the  new  system  will  interface  with  other  systems; 
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How  this  particular  effort  fits  into  any  overall  long 
range  system  development  plan;  and 


•  What  new  technology  the  effort  intends  to  demonstrate. 
The  constraints  identify: 

•  Technology  limitations; 

•  Schedule; 

•  Funding; 

•  Physical  configuration  restrictions; 

•  Political  restrictions;  and 

•  The  Statement  of  Work  itself. 

The  functional  requirements  define  the  functions  that  will 
meet  the  stated  goals  within  the  stated  constraints,  including 
the  inputs,  controls,  outputs,  and  mechanisms  associated  with 
each  function. 

Step  2  -  Preliminary  Design 

The  preliminary  design  establishes  a  design  concept  that 
meeT  *h?  n*>eHc  ider»f  if  *  in  th*  reouirements  analysis.  We  will 

conduct  this  design  process  early  in  Phase  2  of  the  project  and 
submit  it  for  ARI's  approval.  Once  the  design  concept  has  been 
established,  we  can  create  a  software  development  plan.  This 
plan  will  be  based  upon  the  level  of  effort  and  resources  re¬ 
quired  to  implement  the  design  concept.  Although  a  preliminary 
design  concept  has  already  been  proposed,  the  final  concept  may 
vary  with  actual  user  requirements.  The  software  development 
plan  includes  a  work  plan  specifying  the  tasks  that  must  be 
conducted  and  the  order  in  which  they  must  be  performed,  the 
computer  hardware  and  software  resources  needed  to  develop  the 
system,  and  a  detailed  work  schedule. 
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DRC  wi ’ ’  use  an  automated  program  evaluation  and  review 
techniq»>-  ( PERT )  to  create  and  execute  the  software  development 
plan.  The  PERT  is  an  ideal  technique  for  this  project  since  it 
snows  not  only  when  each  task  is  completed,  but  also  how  the 
tasks  interrelate.  The  latter  capability  is  extremely  important 
in  this  effort  because  each  segment  of  the  DSS  is  dependent  upon 
the  others. 

We  will  use  an  IBM  PC  (or  other  computer  if  desired  by  the 
COR)  to  create  the  software  development  plan  and  conduct  the  PERT 
analysis.  Using  a  computer  in  this  subtask  is  critical  since  the 
software  development  plan  and  the  PERT  network  are  complicated 
and  must  be  updated  continually  throughout  the  rest  of  the 
project . 

The  final  products  of  this  subtask  are  a  Preliminary  Design 
Technical  Report  and  the  Software  Development  Plan.  Although  the 
contract  does  not  require  a  Preliminary  Design  Technical  Report, 
it  is  very  important  in  the  system  development  process.  Step  2 
consists  cf  the  following  six  activities: 

Activity  One  -  Prepare  Preliminary  Design 

The  preliminary  design  effort  requires  MPT  research,  system 
engineering,  and  ADP  experience  to  generate  a  system  design 
that  will  satisfy  the  user  requirements.  First,  the  project 
team  translates  the  requirements  into  the  outputs  the  system 
needs  and  develops  the  logic  in  arriving  at  these  outputs. 
The  process  then  dictates  the  input  information  needed  to 
support  the  process.  Next,  the  team  establishes  the  ana¬ 
lytical  procedures  to  support  the  process.  Finally,  in 
order  to  define  what  resources  will  be  required,  the  team 
establishes  the  ADP  procedures  (i.e.,  data  base  management. 
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general  processing  needs,  etc.)*  The  results  of  the  design 
are  documented  in  a  Preliminary  Design  Technical  Report  tha 
the  COR  and  the  technical  members  of  the  development  group 
must  approve. 

Activity  Two  -  Develop  Task  Plan 

The  development  of  the  Task  Plan  (Software  Development  Plan 
consists  of  determining  the  required  tasks,  the  resources 
needed  to  accomplish  these  tasks,-  and  a  work  schedule.  In 
this  activity  we  prepare  the  Work  Plan,  which  describes  the 
work  to  be  accomplished  and  the  order  in  which  it  must  be 
accomplished.  The  key  to  developing  a  workable  plan  is 
having  a  detailed  knowledge  of  the  system  development  pro¬ 
cess  and  extensive  experience  in  applying  it  to  varying 
systems.  The  product  of  this  effort  is  a  work  flow  diagram 
that  not  only  identifies  the  individual  tasks,  but  also 
shows  the  interrelationships  among  them.  Figure  A-2  shows 
typical  diagram  for  a  single  program  development  effort. 

Activity  Three  -  Determine  Required  Resources 

The  resources  needed  to  develop  the  system  are  determined 
bared  upon  the  products  identified  in  Activity  One  and  the 
tasks  developed  in  Activity  Two.  These  resources  include 
manpower,  computer  hardware,  computer  operations,  equipment 
facilities,  and  materials.  For  a  computer  development 
project,  this  activity  also  includes  analysis  of  off-the- 
shelf  software  resources  needs  versus  newly  developed 
software  resources. 
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Activity  Four  -  Develop  Work  Schedule 

The  final  activity  in  determining  the  task  plan  is  sche¬ 
duling  the  work  flow  and  the  resources.  The  schedule  must 
reflect  the  level  of  effort  for  each  task,  the  availability 
of  resources  over  time,  and  the  interdependency  of  tasks  and 
•resources.  Trade-offs  may  be  made  in  the  implementation 
schedules  in  order  to  use  the  resources  more  efficiently. 

Activity  Five  -  Conduct  Critical  Path  Analysis 

This  activity  uses  PERT  to  combine  the  three  activities 
above  into  a  single  critical  path  analysis.  Using  a  network 
type  of  algorithm,  the  PERT  procedure  evaluates  the  inter¬ 
dependency  of  work  packages,  the  expected  time  needed  to 
complete  each  task,  and  the  overall  effect  of  varying  these 
times  on  completing  the  project.  The  PERT  process  provides 
an  excellent  tool  for  de'teloping  the  initial  project  devel¬ 
opment  program.  If  properly  maintained  during  the  project, 
PERT  allows  continual  updating  of  the  project  status  and 
rescheduling  of  tasks  to  meet  changing  resource  allocations 
and  varying  completion  dates.  We  intend  to  apply  the  PERT 
process  throughout  the  development  of  the  Decision  Support 
System . 

Activity  Six  -  Prepare  Software  Development  Plan 

The  final  activity  in  this  subtask  is  preparing  the  Software 
Development  Plan.  The  plan,  prepared  in  accordance  with  ARI 
contract  report  guidelines,  contains  the  information  devel¬ 
oped  in  Activities  Two  through  Five  and  forms  the  basis  of 
the  software  development. 
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Step  3  -  Develop  Detailed  Design 


The  detailed  design  transforms  the  design  concept  into  a 
highly  detailed  system  design  that  is  ready  for  computer 
application.  During  this  step,  the  project  team  determines  the 
specific  analytical  procedures  and  data  handling  procedures. 

Based  upon  the  approved  preliminary  design,  the  project  team 
prepares  a  detailed  system  design  and  determines  if  it  is 
necessary  to  develop  new  software  or  acquire  off-the-shelf 
software.  The  software  is  then  developed  or  acquired,  unit 
tested,  and  integrated  with  the  total  system.  The  team 
simultaneously  determines  the  data  requirements,  constructs  the 
data  bases,  and  uses  data  base  management  systems.  Next,  the  ARI 
staff  tests  the  system  to  ensure  that  it  meets  the  system-stated 
requirements  before  it  is  accepted.  This  test  includes  the 
review  and  approval  of  the  system  documentation.  Finally,  the 
system  is  implemented  on  the  host  computer,  and  ARI  personnel  are 
trained.  We  propose  to  hold  this  review  at  the  end  of  Phase  2  of 
the  project. 

Detailed  design  consists  of  the  following  ten  activities: 

Activity  One  -  Expand  Preliminary  Design  Concept 

This  activity  expands  the  preliminary  design  concept  devel¬ 
oped  in  Step  2.  The  expanded  version  includes  a  detailed 
description  of  the  analytical  procedures  to  be  employed,  the 
input  needed  to  support  the  procedures,  and  the  output  re¬ 
sulting  from  the  process.  The  detailed  design  is  presented 
to  the  COR  at  the  Critical  Design  Review  ( CDR )  for  approval. 

Once  approved,  the  design  becomes  the  Critical  Design  Base¬ 
line.  The  detailed  design  specifications  are  considered  the 
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"buiid-to"  specif ication  and  are  written  at  the  level  where 
a  programmer  can  write  code  or  adapt  an  existing  software 
package  without  further  design.  During  this  activity,  the 
team  establishes  the  configuration  management  program  and 
the  guidelines  for  the  system  documentation. 

Activity  Two  -  Analyze  Data  Requirements 

The  detailed  design  conducted  on  Activity  One  determines  the 
precise  inputs  required  by  the  system,  and  the  form  of  the 
output  activity  2  is  a  data  requirements  analysis,  done  to 
determine  the  data  elements  that  support  the  input  and 
output.  The  analysis  defines  the  source  of  the  data,  the 
means  of  collecting/transferring  them  to  the  system,  and 
their  management  within  the  system.  An  important  product  of 
this  activity  is  the  Data  Element  Dictionary,  which 
describes  each  data  element  by  type,  source  function,  and 
storage  requirements. 

Activity  Three  -  Locate  Existing  Software 

A  key  feature  of  our  software  development  plan  is  to  use 
existing  software  whenever  possible.  This  approach 
drastically  reduces  the  time  and  risk  associated  with 
developing  new  software,  since  off-the-shelf  software  has 
already  been  tested  and  proven  effective.  In  this  activity, 
we  search  for  compatible  off-the-shelf  software  that  meets 
our  design  specifications.  We  may  use  available  analytical 
programs  as  well  as  specialized  models  already  developed 
(Micro  SAINT).  We  also  plan  to  use  one  of  the  many  data 
base  management  systems  (DBMSs)  on  the  market  to  manage  the 
data  bases.  Hopefully,  much  of  this  software  can  be  used  as 
is.  If  the  software  does  not  meet  our  precise  needs,  we 
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will  adapt  the  existing  code  rather  than  attempt  to  build 
software  from  the  ground  up.  We  will  build  software  only  if 
we  cannot  find  any  existing  programs,  and  we  will  ensure 
that  the  software  we  build  is  transportable  and  easily 
adaptable  to  similar  problems. 

Activity  Four  -  Develop  Unit  Test  Code 

The  most  time-consuming  task  in  software  development  is 
creating  or  revising  computer  code  and  unit  testing  it.  We 
will  develop  code  using  common,  standard,  high-level 
programming  languages  that  are  highly  reliable  and 
t'ansportable  (such  as  FORTRAN  and  COBOL).  All  code, 
whether  developed  initially  by  us,  adapted  from  code  we  have 
acquired,  or  taken  directly  off-the-shelf,  must  be  unit 
tested  to  ensure  that  it  meets  the  requirements  Activity  One 
established.  The  unit  test,  using  test  data  generated  for 
the  unique  purpose  of  unit  testing,  completes  the  computer 
development  step  and  establishes  the  Segment  Software  Test 
Baseline. 

Activity  Five  -  Conduct  Integration  Test 

The  completed  software  segments  are  then  tested  as  an 
integrated  package  using  live  data  collected  by  the  data 
base  development  effort.  Once  the  Quality  Assurance  (QA) 
Testing  Team  has  determined  that  the  code  meets  the  system 
requirements  established  in  Subtask  2.1  and  the  standards  of 
both  the  company  and  the  Government,  then  the  software 
becomes  the  Software  System  Test  Baseline. 
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Activity  Six  -  Conduct  Acceptance  Test 


While  the  computer  code  is  being  developed  and  tested,  the 
programming  team  prepares  the  system  documentation  according 
to  the  standards  identified  in  Activity  One.  During  the 
integration  testing,  the  documentation  is  reviewed  for 
completeness,  accuracy,  and  conformity  to  the  standards. 
After  the  QA  team  approves  the  documentation  (as  well  as  the 
integrated  software),  the  system  goes  to  the  COR  for 
acceptance  testing.  This  time  the  COR  will  evaluate  the 
software's  accuracy,  suitability,  and  usability  using  the 
following  definitions: 

Accuracy ,  or  the  model's  ability  to  produce  accurate 
results,  is  measured  in  terms  of  technical  validity. 
Technical  validity  requires  that  the  model's  assump¬ 
tions  represent  the  real  world  it  is  intended  to 
simulate,  that  the  data  used  in  the  model  is  correct, 
that  the  mathematical  formulations  are  appropriate  and 
correct,  and  that  the  errors  between  the  actual 
outcomes  and  predicted  outcomes  do  not  result  from 
modeling  parameters. 

Suitability,  or  the  extent  to  which  the  model's  outputs 
satisfy  the  user's  requirements,  is  verified  using  a 
requirements  traceability  matrix.  During  the  require¬ 
ments  analysis,  the  detailed  requirements  are  listed  in 
a  traceability  matrix. 

As  the  development  continues,  the  requirements  are 
traced  through  the  preliminary  design,  the  detailed 
design,  the  computer  code,  and  the  documentation.  At 
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the  Acceptance  Test,  the  matrix  is  reviewed  to  identify 
each  requirement  in  each  development  step  and  its 
presence  in  the  final  software  and  documentation. 

Usability,  or  the  model's  usefulness  in  a  realistic 
environment,  is  measured  by  the  model's  ability  to 
conduct  the  analyses  for  which  it  was  developed.  This 
final  criterion,  typically  called  "operational 
validity",  assesses  the  model's  ability  to  produce 
results  acceptable  to  the  user. 

Since  the  model  cannot  predict  the  real  environment 
perfectly,  operational  validity  must  conclude  whether 
the  model's  use  is  appropriate  for  the  observed  and 
expected  errors. 

Activity  Seven  -  Implement  Software/Train  Users 

The  completed  and  approved  system  is  then  implemented  on 
ARl's  computer.  To  ensure  its  success,  we  will  provide  a 
formal  training  program  for  ARI.  Build  around  the  system 
documentation,  the  training  describes  the  system's  opera¬ 
tion,  preparation  of  the  inputs,  and  use  of  the  output.  The 
training  will  include  extensive  hands-on  training  at  the 
computer  terminal,  where  most  operations  occur. 

Activity  Eight  -  Integrate  Software 

The  analytical  tools  developed  become  part  of  the  integrated 
(MPT)2  system,  through  the  integration  of  their  operations 
and  data.  During  this  activity,  the  integration  between 
this  model  and  the  decision  support  system  tools  (especially 
the  Planner/Estimator  and  the  Executor/Interpreter)  is 
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established.  Both  the  information  required  and  the  means  of 
integration  are  described  below. 

Activity  Nine  -  Evaluate  Software 

After  our  contract  has  ended,  the  using  community  will  eval¬ 
uate  the  software.  This  activity  represents  a  separate 
function  from  the  development  process.  It  allows  the  user 
to  become  familiar  with  the  system,  while  the  developer  is 
close  at  hand  to  provide  assistance  and,  if  desired,  revise 
the  model  to  correct  any  deficiencies  in  the  original  re¬ 
quirements  and  design.  This  activity  is  conducted  in  paral¬ 
lel  with  Activity  Ten,  which  provides  similar  support. 

Activity  Ten  -  Provide  Operational/Maintenance  Support  on  (MPT)2 

After  tne  software  is  accepted  by  the  user,  a  ;.,aiiictr»ance 
organization  must  continue  to  provide  operational  and 
maintenance  support.  This  support  includes  demonstrating 
how  to  use  the  software  in  an  operational  environment, 
helping  the  user  apply  the  software  to  actual  operational 
problems,  and  maintaining  the  software  on  the  host 
computer.  Any  revisions  made  to  the  computer  code  during 
this  time  will  be  reflected  in  the  documentation. 
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APPENDIX  B— FORMATS  FOR  ARMY  REQUIREMENTS  DOCUMENTS 

OPERATIONAL  AND  ORGANIZATIONAL  PLAN  (OAO  PLAN) 

OPERATIONAL  AND  ORGANIZATIONAL  PLAN  (040  Plan)  FORMAT 


The  040  Plan  describes  how  a  system  will  be  Integrated  Into  the  force 
structure,  deployed,  operated,  and  supported  In  peacetime  and  wartime. 
The  concept  establishes  required  readiness  objectives  and  Is  the  basis 
for  Integrated  Logistic  Support  planning.  Initially,  the  plan  should, 
as  a  minimum,  describe  any  deficiencies  which  were  Identified  In  the  HAA 
and  any  constraints  applicable  to  systems  development. 

I.  Purpose  *  Describe  the  need  for  an  operational  capability  to 

defeat  the  threat  and  eliminate  an  operational  defi¬ 
ciency.  State  where  In  the  KAA  the  deficiency  Is 
Identified  and  how  the  need  was  developed  from  the 
described  deficiency.  (The  need  should  be  stated  In 
broad  characteristics  only  (e.g.,  a  capability  Is 
needed  to  defeat  enemy  armor  at  "x"  kilometers)). 

II.  Threat/ 

Deficiency  -  Describe  the  threat  to  be  countered  and  the  opera¬ 

tional  deficiency  to  be  eliminated. 

•III.  Opera¬ 
tional  Plan  -  Oescrlbe  how,  what,  when,  and  where  the  system  will 

be  employed  on  the  battlefield  and  how  It  will 
Interface  with  other  systems  (attach  Operational 
Mode  Suimary/MI sslon  Profile  as  an  annex).  Communi¬ 
cations  support  requirements  should  be  addressed. 

•IV.  Organ¬ 
izational 

Plan  -  Discuss  the  type  units  that  will  employ  and  support 

the  system  and  when  appropriate,  the  system! s)  to  be 
replaced.  (When  the  system  Is  decided  on,  Include 
the  number  of  systems  estimated  to  be  provided  each 
type  unit).  This  plan  will  support  preparation  of 
the  BOIP,  the  Integrated  Logistic  Support  Plan  and 
Identification  of  key  ancillary  Items. 

•V.  Person¬ 
nel  Impact  -  Design  of  the  system  should  consider  personnel 

skills  available  to  operate  and  maintain  the  system. 
Generation  of  new  MOS  should  be  avoided  where  pos¬ 
sible.  (When  the  system  Is  decided  on.  Include  an 
estimate  of  the  number  of  people  ard  skills  esti¬ 
mated  to  operate  and  maintain  the  equipment,  by  type 
unit.)  This  plan  will  support  preparation  of  the 
Tentative  Qualitative  and  Quantitative  Personnel 
Requirements  Information  (TQQPRI ),  the  Personnel 
Support  Plan,  and  assist  In  the  LSA  process. 
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Appendix  B  (Continued) 

OPERATIONAL  AND  ORGANIZATIONAL  PLAN  (O&O  PLAN) 


OPERATIONAL  ANO  ORGANIZATIONAL  PLAN  (0&0  Plan)  FORMAT 

(continued ) 


*V I .  Train¬ 
ing  Impact 


•VII.  Logis¬ 
tics  Impact 


Design  of  the  equipment  should  consider  type  and 
extent  of  training  required.  (When  the  system  Is 
decided  on,  discuss  the  type  and  amount  of  training 
requ'red  and  the  need  for  training  devices  and  simu¬ 
lators.)  This  plan  will  support  preparation  of  the 
Training  Support  Plan. 


System  must  be  supportable  by  the  Standard  Army 
Logistics  System  and  use  standard  tools  and  TMDE. 
(When  the  system  is  decided  on,  the  proposed  levels 
of  maintenance,  support  concept.  Test,  Measurement, 
and  Diagnostic  Equipment  (TMDE),  Automatic  Test 
Equipment  (ATE),  and  Built-in  Test  Equipment 
(BITE)  concepts  will  be  discussed.)  This  plan  will- 
support  preparation  of  the  Integrated  Logistic  Sup¬ 
port  Plan. 


Complete  information  for  these  paragraphs  may  not  be  available  when 
the  initial  0&0  Plan  1$  prepared. 
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Appendix  B  (Continued) 


JUSTIFICATION  FOR  MAJOR  SYSTEM  NEW  START(JMSNS) 


JUSTIFICATION  FOR  MAJOR  SYSTEM  NEW  START  (JMSNS)  FORMAT 


Prepare  JMSNS  In  the  format  shown  below.  Do  not  exceed  3  pages,  includ¬ 
ing  annexes.  Identify  any  supporting  documentation. 

A.  Defense  Guidance  Element.  Identify  the  element  of  Defense  Guidance 
to  which  the  system  responds. 

B.  Mission  and  Threat.  Identify  the  mission  area  (numbers  and  title) 

and  describe' the  role  of  the  system  In  the  mission  area.  Olscuss 
the  DIA-val Idated  projected  threat  and  the  shortfalls  of  existing 
systems  In  meeting  the  threat.  Comment  on  the  tiding  of  the  need 

and  the  general  priority  of  this  system  relative  to  others  In  this 
mission  area.  The  TRAOOC  school  e>*  Integrating  Center  must  obtain 
a  DIA-val Idated  threat  from  INSCOM  early  so  as  not  to  delay  JMSNS 
preparation.  The  classification  should  be  as  low  as  possible; 
NOFORN  data  should  not  be  Included.  DIA  threat  documentation 
should  be  referenced  In  lieu  of  higher  classification.  If  the  need 
Is  not  threat  driven  describe  the  basis  for  the  need  (e.g.,  cost 
savings). 

C.  Alternative  Concepts.  Describe  the  alternatives  which  will  be 
considered  (Including  product  Improvements)  and,  when  appropriate, 
the  alternative  selected,  the  reasons  for  rejecting  those  that  have 
not  been  selected,  and  any  further  tradeoffs  that  remain  for  the 
selected  system. 

D.  Technology  Involved.  Discuss  maturity  of  the  technology  planned 
for  the  selected  system  design  and  manufacturing  processes,  when 
appropriate,  with  particular  emphasis  on  remaining  areas  of  risk. 

E.  Funding  Implications.  Provide  gross  estimates  of  total  RDT&E  cost, 
total  procurement  cost,  unit  cost  and  life-cycle  cost.  Discuss 
affordabl 1 Ity.  See  Appendix  D,  this  Handbook,  for  funding  format. 

F.  Constraints.  Describe,  as  applicable,  key  boundary  conditions  for 
satisfying  the  need,  such  as  survivability;  loalstlcs,  manpower  and 
personnel  constraints  In  both  quantity  and  quality;  standardization 
or  interoperability  within  NATO  or  other  DOD  Components;  and  criti¬ 
cal  materials  and  Industrial  base  required. 

G.  Acquisition  Strategy.  Provide  sunnary  of  salient  elements  of  pro¬ 
posed  acquisition  strategy  --  program  structure,  competition,  con¬ 
tracting,  etc. 
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Appendix  B  (Continued) 


LETTER  OF  AGREEMENT  (LOA) 


LETTER  OF  AGREEMENT  (LOA)  FORMAT 

The  Letter  of  Agreement  (LOA)  will  be  in  the  format  below.  tlmlt  In¬ 
formation  to  that  necessary  for  a  HQDA  decision.  The  basic  document 
should  not  exceed  four  pages.  In  the  LOA,  use  less  detail  and  broader 
performance  bands  than  In  the  ROC,  JSOR,  LR,  and  TOR.  Terms  in  each 
paragraph  of  the  LOA  will  evolve  into  more  specific  terms  in  the  ROC,  LR 
and  TOR.  Include  in  the  LOA  all  alternative  system  concepts  recommended 
for  demonstration  and  validation. 

1.  TITLE 

a.  Give  a  descriptive  title  for  the  program. 

b.  CARDS  reference  number. 

2.  NEED/THREAT.  State  what  is  needed.  Briefly  describe  the  threat  and 
operational/training  deficiency  need  for  the  system.  Include  the  ene¬ 
my’s  capability  to  detect.  Identify,  locate,  avoid,  suppress,  destroy, 
or  otherwise  counter  the  system.  Describe  the  responsive  threat  over 
time  to  support  evolutionary  development  when  applicable. 

3.  TIMEFRAME  AND  IOC.  State  the  timeframe  In  which  the  new  or 
Improved  system  Is  needed. 

4.  OPERATIONAL  &  ORGANIZATIONAL  PLAN:  In  «  brief  paragraph  state  -- 

a.  How  the  equipment  will  be  used; 

b.  Geographical  areas  of  use; 

c.  weather  and  climatological  factors  to  be  considered  during 
equipment  operations; 

d.  Battlefield  conditions  (such  as  ECM,  smoke,  and  dust)  In  which 
the  system  will  operate;  and 

e.  The  type  of  units  that  will  use  and  support  the  equipment. 
Attach  the  mission  profile  to  the  LOA  as  an  Annex. 

5.  ESSENTIAL  CHARACTERISTICS.  Oescrlbe  only  main  operational  features 
of  the  system.  Included  are  counter-countermeasure  capabilities, 
health,  physical  security,  safety  and  human  factors  engineering  require¬ 
ments,  and  reliability,  availability,  and  maintainability  (RAM)  require¬ 
ments.  Performance  must  be  responsive  to  battlefield  environmental 
conditions  of  continuous  combat  (such  as  full  ECM,  smoke,  aerosols, 
rain,  fog,  haze,  and  dust). 
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Appendix  B  (Continued) 


LETTER  OF  AGREEMENT  (LOA) 


LETTER  OF  AGREEMENT  (LOA)  FORMAT 
(continued ) 

Express  performance  and  reliability  characteristics  in  bands  of  perfor¬ 
mance.  Those  which  are  not  suitable  for  banding  will  be  stated  as 
single  values.  During  development,  contnerci  al ,  other  service,  NATO,  or 
other  allied  nation  characterl sties  of  existing  or  programed  systems 
should  be  considered  for  inclusion.  This  will  be  with  a  view  toward 
establishing  a  basis  for  interoperabl 1 ity ,  co-production,  or  standardi¬ 
zation.  Bands  of  performance  should  be  flexible  enough  to  consider 
competing  systems  of  other  services  or  allied  nations.  Stated  bands  of 
performance,  or  single  value  character! sties  will  be  adjusted  only  after 
the  combat  and  materiel  developers  agree  that  such  changes  are  neces¬ 
sary.  DCSOPS  will  approve  changes  for  documents  previously  approved  by 
DC  SOPS .  The  requirements  and  provisions  for  the  following  must  be 
considered. 


a . 

Interoperabl 1 ity; 

b. 

Continuity  of  Oper 

atlons  ( CONOPS ) ; 

c. 

Secur 1 ty; 

d. 

Reliability,  avai 

1 abi 1 1 ty , 

and  mal 

n  t  a  1 

nab  1 1 1 ty  (RAM)  derived 

miss 

ion  performance  par 

ameters . 

e . 

Standardization, 

including 

conmona 

Ity 

for  hardware  and  soft- 

to  w 

h i ch  the  system  «i 1 

1  adhere, 

f  . 

ionnuc lear/nuc 1  ear 

Survi vab i 1 i ty ; 

NBC 

con  t ami n at  1  on /dec  on- 

Lamination  survivability, 

g.  Indi vidual /col lecti ve  protection  equipment; 

h.  Adverse  weather  and  reduced  visibility  conditions  (smoke  and 
obscurants)  operations,  and  military  operations  on  urbanized  terrain 
(MOUT)  where  applicable; 

1.  Communications; 

j.  Operation  transported  1 1 ty,  such  as:  transportable  in  C-141 
type  aircraft  requiring  not  rtore  than. .. .hours  tuardown  and  ... .hours 
setup  by  operator  and  crew,  etc. 

6.  TECHNICAL  ASSESSMENT.  In  the  LOA,  divide  this  paragraph  into 
operational,  technical,  logistics,  training,  and  manpower  subparagraphs. 
In  each,  describe  what  the  combat  and  materiel  developers,  logistician, 
trainer,  and  personnel  admini strator  must  do  to  produce  the  total  sys¬ 
tem.  Include  a  listing  of  major  events  and  dates. 
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Appendix  B  (Continued) 


LETTER  OF  AGREEMENT  (LOA) 


LETTER  AGREEMENT  OF  AGREEMENT  (LOA)  FORMAT 
(continued) 


7.  LOGISTICS  SUPPORT  PLAN.  8rief1y  describe  the  logistics  support 
plan.  The  logistics  support  plan  will  be  available  for  evaluation  during 
OT  I. 


8.  TRAINING  ASSESSMENT.  Discuss  the  need  for  system  training  devices. 
When  required  Include  description  as  an  annex.  (See  p.  6.20  for  for¬ 
mat.)  New  Equipment  Training  (NET),  operator  and  maintenance  personnel 
training,  and  technical  manuals  and  training  material  requirements  will 
be  stated  In  terms  of  needs  for  both  the  Institution  and  unit  training 
levels.  The  training  support  plan  will  be  available  for  evaluation 
during  OT  I. 

9.  MANPOWER/FORCE  STRUCTURE  ASSESSMENT.  Estimate  manpower  require¬ 
ments  per  system,  using  unit,  and  total  Army  by  component  (Active,  ARNG, 
USAP).  Identify  manpower  savings  resulting  from  replaced  system:,  If 
any.  Include  a  statement  to  require  an  assessment  of  alternatives  to 
reduce  manpower  requirements  and  an  assessment  of  force  structure  Impli¬ 
cations  resulting  from  system  Inclusion  In  the  total  force  by  component. 
If  the  force  structure  assessment  exceeds  current  programed  force 
structure  levels  then  Identification  of  force  structure  tradeoffs  within 
mission  area  or  mission  elements  are  required.  Tradeoffs  analyses  are 
addressed  to  the  degree  necessary  to  bring  the  force  structure  assess¬ 
ment  within  current  programing  levels.  If  possible.  The  personnel 
support  plan  will  be  available  for  evaluation  during  OT  I. 

10.  RATIONALIZATION.  STANDARDIZATION,  INTEROPERABILITY.  Discuss  other 
Services,  NATO,  and  other  foreign  Interest  In  the  program.  Identify 
Similar  programs  contemplated  by  other  services,  NATO  or  other  allies. 

11.  LIFE  CYCLE  COST  ASSESSMENT.  See  appendix  1. 


\Z  MILESTONE  SCHEDULE.  A  listing  of  significant  events  with  dates  to 
occur  between  approval  of  the  LOA  and  next  scheduled  ml II es tone  review. 
The  following  should  be  Included:  LOA  approval,  DT/OT/other  test  (Mar¬ 
ket/User  Survey  for  OTS),  and  next  scheduled  milestone  review. 

APPENDIX  1  -  Life  Cycle  Cost  Assessment  -  Provide  life-cycle  costs  using 
mainly  sugary  parametric  estimating  techniques.  State  the  major  1  fe- 
cycl®  phases  of  140,  investment,  and  operation  and  support.  A  so  Include 
the  design  to  cost  goals.  As  much  as  possible,  show  the  estimated  cost 
of  major  Ue<s  o r  components  below  the  system  level.  These  *  '  jj,.. 
be  consistent  with  the  Materiel  System  Requirements  Specification  (MSRS) 

and  Gaseline  Cost  Estimate  (BCE). 
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Appendix  B  (Continued) 

LETTER  OF  AGREEMENT  (LOA) 


LETTER  of  AGREEMENT  (LOA)  FORMAT 
(continued) 

ANNEX  A  -  Coordination.  List  all  major  commands,  other  Services,  allied 
nations,  and  activities  with  whom  the  LOA  was  coordinated.  Provide  full 
rationale  for  nonacceptance  of  comments,  If  any. 

ANNEX  B  -  Operational  Mode  Sumnary /Mission  Profile  Annex.  List  tasks 
and  conations  for  frequency  and  urgency  viewed  for  system  employment  In 
military  operations.  The  mission  profile  Is  logically  derived  from  the 
040  Plan.  It  provides  the  starting  point  for  developing  the  system 
characteristics.  See  p.  S.23  for  format  for  mission  profile. 

ANNEX  C  -  COEA  Annex.  Executive  summary  of  the  COEA.  Classify  as  re¬ 
quired.  Withdraw  after  HQ  TRADOC  approval  of  the  LOA  and  handle  as  a 
separate  document  for  transmittal  as  needed. 

ANNEX  0  -  Rationale  Annex.  Support  various  characteristics  stated  In 
the  LOA.  This  provides  an  audit  trail  and  rationale  for  determining  how 
the  characteristics  were  derived. 

ANNEX  £  -  RAM  Rationale  Annex.  Executive  summary  of  the  RAM  Rationale 
Report.  Support  the  stated  RAM  characteristics  with  a  logical  argument 
that  begins  with  the  task  frequency,  conditions  and  standards  described 
and  analyzed  In  the  MAA.  This  prov'des  an  audit  trail  and  rationale  for 
determining  hov/  the  characterl sties  were  derived.  TRADOC/DARCOM 
Pamphlet  70-11  contains  guidance  on  the  preparation  of  both  the  RAM 
Rationale  Report  and  the  RAM  Rationale  Annex. 

ANNEX  F  -  Training  Oevlces.  When  required,  include  description  of  need¬ 
ed  training  devices  In  format  on  p.  6.20.  A  separate  annex  Is  required 
for  each  training  device. 

NOTES: 

1.  All  annexes  will  accompany  the  LOA  until  It  has  completed  TRADOC 
and  OARCOM  staffing. 

2.  Send  A,  B,  and  F  with  the  LOA  when  forwarded  to  HQDA  for  appro¬ 
val  . 
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REQUIRED  OPERATIONAL  CAPABILITY  (ROC) 


REQUIRED  OPERATIONAL  CAPABILITY  (ROC)  FORMAT 


Tht  Required  Operational  Capability  (ROC)  1*  In  the  format  below.  Limit 
Information  to  that  necessary  for  a  HQDA  decision.  Tha  basic  document 
should  not  axceed  four  pages. 

1.  TITLE 

a.  Give  a  descriptive  title  for  tha  program. 

b.  CARDS  reference  number. 

2.  NEED/THREAT.  Briefly  describe  the  operatlonal/tralnlng  deficiency 
need  for  the  system  and  the  reactive  threat  to  the  system.  Include  the 
enemy’s  capability  to  detect.  Identify,  locate,  avoid,  suppress,  des¬ 
troy,  or  otherwise  counter  the  system.  Describe  the  responsive  threat 
over  time  to  support  evolutionary  development  when  applicable. 

3.  TIMEFRAME  AND  IOC.  State  the  IOC  date  Including  IOCs  for  succes¬ 
sive  evolutionary  models,  when  appropriate. 

4.  OPERATIONAL  AND  ORGANIZATIONAL  PLAN  (040  Plan).  In  a  brief  para¬ 
graph  state: 

a.  How  the  equipment  will  be  used; 

b.  Geographical  areas  of  use; 

c.  Weather  and  climatological  factors  to  be  considered  during 
equipment  operations; 

d.  Battlefield  conditions  (such  as  ECM,  smoke,  and  dust)  In  which 
the  system  will  operate;  and 

e.  The  type  of  units  that  will  use  and  support  the  equipment. 

5.  ESSENTIAL  CHARACTERISTICS.  Describe  only  main  operational  features 
of  the  system.  Included  are  counter-countermeasure  capabilities, 
health,  safety  and  human  factors  engineering  requirements,  and  reliabi¬ 
lity,  availability,  and  maintainability  (RAM).  Performance  must  be  re¬ 
sponsive  to  battlefield  environmental  conditions  of  continuous  combat 
(such  as  full  ECM,  smoke,  aerosols,  rain,  fog,  haze,  and  dust). 

Express  performance  and  reliability  characteristics  in  bands  of  perfor¬ 
mance.  Those  which  are  not  suitable  for  banding  will  be  stated  as 
single  values.  During  development,  cormerclal,  other  Service,  NATO,  or 
other  allied  nation  characteristics  of  existing  or  programed  systems 
should  be  considered  for  Inclusion  with  a  view  toward  establishing  a 
basis  for  Interoperability,  co-production,  or  standardization.  Bands  of 
performance  should  be  flexible  enough  to  consider  competing  systems  of 
other  Services  or  allied  nations.  Stated  bands  of  performance,  or 
single  value  characteristics  are  adjusted  only  after  the  combat  and 
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REQUIRED  OPERATIONAL  CAPABILITY  (ROC) 


REQUIRED  OPERATIONAL  CAPABILITY  (ROC)  FORMAT 
(continued) 


materiel  developers  agree  that  changes  are  necessary.  DCSOPS  will 
approve  changes  for  documents  previously  approved  by  DCSOPS.  The  re¬ 
quirements  and  provisions  for  the  following  must  be  considered: 

a.  Interoperability; 

b.  Continuity  of  Operations  (CONOPS); 

c.  Security; 

d.  Reliability,  availability,  and  maintainability  (RAM)  derived 
from  mission  performance  parameters; 

e.  Standardization,  Including  commonality  for  hardware  and  soft¬ 
ware  to  which  the  system  will  adhere; 

f.  Nuclear  survivability;  NBC  contamination  survivability; 

g.  Indlvldual/col lectl ve  protection  equipment; 

h.  Adverse  weather  and  reduced  visibility  (smoke  and  obscurants) 
operations,  and  military  operations  on  urbanized  terrain 
(MOUT)  where  applicable; 

1.  Communications. 

j.  Operation  transportability  requirements,  such  as:  transport¬ 
able  In  C-141  type  aircraft  requiring  not  more  than. .. .hours 
teardown  and.... hours  set  by  operator  and  crew;  etc. 

k.  P21 

6.  TECHNICAL  ASSESSMENT.  In  the  ROC,  include  a  brief  paragraph  about 
the  technical  effort  required.  Address  major  areas  for  full  scale 
development  In  terms  of  scope,  technical  approach,  and  associated  risks 
In  high,  medium,  low,  or  similar  categories.  For  NDI  Items,  briefly 
outline  completed  or  planned  market  survey  efforts  and/or  military 
suitability  evaluations. 

7.  LOGISTICS  SUPPORT  PLAN.  Briefly  describe  the  logistics  support  con¬ 
cept.  The  logistics  support  package  will  be  tested  during  OT  II. 

8.  TRAINING  ASSESSMENT.  Discuss  the  need  for  system  training  devices. 
When  required.  Include  description  as  an  anne*  to  the  ROC.  (See  p.  6.16 
for  format.)  New  equipment  training  (NET)  operator  and  maintenance  per¬ 
sonnel  training,  and  technical  manuals  and  training  materiel  require¬ 
ments  will  be  stated  In  terms  of  needs  for  both  Institution  and  unit 
training  levels.  The  training  support  package  will  be  tested  during  OT 
II. 

0 

9.  MANPOWER/FORCE  STRUCTURE  ASSESSMENT.  Estimate  manpower  requirements 
per  system,  using  unit,  and  total  Army  by  component  (Active,  AANQ, 
USAR).  Identify  manpower  savings  resulting  from  replaced  Systems,  .  If 
any.  Include  a  statement  to  require  an  assessment  of  alternatives  "to'rv 
reduce  manpower  requirements  and  an  assessment  of  force  Structure  lapi 1 f 
cations  resulting  from  system  inclusion  In  the  total  force  by  component. 
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REQUIRED  OPERATIONAL  CAPABILITY  (ROC) 


REQUIRED  OPERATIONAL  CAPABILITY  (ROC)  FORMAT 
(continued) 


If  the  force  structure  assessment  exceeds  current  programed  force 
structure  levels  then  Identification  of  force  structure  tradeoffs  within 
mission  area  or  mission  elements  Is  required.  Tradeoffs  analysis  are 
addressed  to  the  degree  necessary  to  bring  the  force  structure  assess* 
ment  within  current  programing  levels.  If  possible.  The  personnel 
support  package  will  be  tested  during  OT  II . 

10.  STANDARDIZATION,  INTEROPERABILITY.  Discuss  other  Service,  NATO,  and 
other  foreign  Interest  In  the  program.  Identify  similar  programs  con¬ 
templated  by  other  Services,  NATO  or  other  allies. 

11.  LIFE  CYCLE  COST  ASSESSMENT.  See  appendix  1. 

12.  MILESTONE  SCHEDULE.  A  listing  of  significant  events  with  dates  to 
occur  between  approval  of  the  ROC  and  next  scheduled  milestone  review. 
The  following  should  be  Included:  ROC  approval,  DT/OT/other  test  (Mar¬ 
ket/User  Survey  for  OTS),  and  next  scheduled  milestone  review. 

APPENDIX  1  -  Life-cycle  Cost  Assessment.  Provide  life-cycle  costs 
using  mainly  summary  parametric  estimating  techniques.  State  the  major 
life  cycle  phases  of  RAD,  Investment,  and  operation  and  support.  Also 
Include  the  design-to-cost  goals.  As  much  as  possible,  show  the  esti¬ 
mated  cost  of  major  Items  or  components  below  the  system  level.  (These 
data  should  be  consistent  with  the  Materiel  System  Requirements  Specifi¬ 
cation  (MSRS)  and  Baseline  Cost  Estimate  (BCE).  (See  app  D,  p.  D.7, 
this  handbook,  for  format). 

ANNEX  A  -  Coordination.  List  all  major  commands,  other  Services,  allied 
nations  and  activities  with  whom  the  ROC  was  coordinated.  Provide  full 
rationale  for  nonacceptance  of  conments,  If  any. 

ANNEX  B  -  Operational  Mode  Summary /Mission  Profile  Annex.  list  tasks 
and  conditions  for  frequency  and  urgency  viewed  for  system  employment  In 
military  operations.  The  mission  profile  Is  logically  derived  from  the 
operatlonal/tralnlng  concept.  It  provides  the  starting  point  for  devel¬ 
oping  the  system  characteristics. 

ANNEX  C  -  COEA  Annex.  Executive  suttmary  of  the  COEA.  Classify  as  re¬ 
quired.  Withdraw  after  HQ  TRADOC  approval  of  the  ROC  and  handle  as  a 
separate  document  for  transmittal  as  needed. 

ANNEX  D  -  Rationale  Annex.  Support  various  characteristics  stated  In 
the  ROC.  this  provides  an  audit  trail  and  rationale  for  determining  how 
the  characteristics  were  derived. 


E3-B-10 


Appendix  B  (Continued) 


REQUIRED  OPERATIONAL  CAPABILITY  (ROC) 


REQUIRED  OPERATIONAL  CAPABILITY  (ROC)  FORMAT 
(continued) 


ANNEX  E  -  RAM  Rationale  Annex.  Executive  sunnary  of  the  RAM  Rationale 
Report.  Support  the  stated  RAM  characteristics  with  a  logical  argument 
that  begins  with  the  task  frequency,  conditions,  and  standards  described 
and  analyzed  In  the  Mission  Area  Analysis  (MAA).  This  provides  an  audit 
trail  and  rationale  for  determining  how  the  characteristics  were  deriv¬ 
ed.  TRADOC/OARCOfl  Pamphlet  70-11  contains  guidance  on  the  preparation  of 
both  the  RAM  Rationale  Report  and  the  RAM  Rationale  Annex. 

ANNEX  F  -  TRAINING  DEVICE  ANNEX.  Include  when  appropriate.  (See  p.  6.20 
for  format.)  A  separate  annex  Is  required  for  each  training  device. 

NOTES:  1.  Send  annex  A  with  each  requirements  document. 

2.  Annex  F  (when  prepared)  must  accompany  the  ROC  to  HQDA  for 
approval  as  a  package. 

3.  Send  the  TBO I P/TQQPRI  with  the  ROC  to  HQDA  for  approval. 
When  the  TBOIP/TQQPRI  are  not  submitted,  the  transmittal  let¬ 
ter  will  contain  a  statement  about  the  projected  submission 
date. 


E3-B-11 


APPENDIX  C  -  TMACS  AND  UNIT  TRAINING 
RESOURCE  PLANNING  PROCESS 


Training  Management  Control  system  (TMACS)  is  a  minicomputer- 
based  system  that  determines  the  cost  of  training  events.  It 
helps  Active  Component  commanders  plan  training,  evaluate  the 
resource  impacts  of  training  events,  and  record  training 
accomplished  and  resources  expended.  TMACS  is  designed  to  be 
operated  by  soldiers  with  no  previous  experience  in  computer 
operation  and  uses  minicomputers  located  at  installation/division 
and  brigade  headquarters.  Battalion  personnel  normally  use  TMACS 
minicomputers  xocated  at  brigade. 

TMACS  uses  three  variables  to  estimate  the  cost  of  training 
events : 

•  The  size  and  type  of  the  par ticipating  organization. 

•  The  cost  factor  of  each  vehicle  and  weapon  used  in  each 
training  event  type.  Cost  factors  are  different  for 
each  unit  or  event  type.  They  are  obtained  from  the 
installation-level  agencies  such  as  the  comptroller  or 
G3.  The  comptroller  is  responsible  for  keeping  the 
cost  factors  current. 

•  The  usage  factor  (miles,  hours,  and  rounds)  of  each 
vehicle  and  weapon  system  used  in  each  type  of  training 
event . 
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Training  events  used  in  estimating  costs  can  come  from  the  unit's 
long-range  and  short-range  planning  calendars.  Examples  of 
battalion-level  events  used  for  estimating  resource  requirements 
include:  individual  weapons  training,  squad  training  and 

evaluation,  crew-served  weapons  training,  and  battalion  Field 
Training  Exercises  (FTXs).  Examples  of  activities  used  to 
estimate  resource  requirements  above  battalion  level  include: 
brigade  Emergency  Deployment  Readiness  Exercises  (EDREs)  and 
brigade,  division,  or  corps  Command  Post  Exercises  (CPXs)  and 
FTXs. 

Output  from  the  TMACS  is  fed  into  the  command  operating  budget 
(COB).  Budget  guidance  received  from  higher  headquarters  goes 
into  the  command  operating  budget.  Commanders  typically  use  the 
COB  input  to  prepare  long-range  plans  for  unit  training 
programs.  This  input  contains  budget  estimates  for  upcoming 
training  events.  The  COB  is  based  on  input  from  battalions  and 
separate  companies  that  are  consolidated  at  higher  levels  between 
March  and  June  of  each  year  and  forwarded  to  Department  of  the 
Army  (DA)  during  July. 

The  COB  is  received  at  DA  after  Congress  has  approved  the  total 
budget.  It  serves  as  the  basis  for  actual  budget  allocations. 

In  both  the  Active  Component  (AC)  and  Reserve  Component  (RC), 
actual  funding  is  received  at  the  start  of  the  fiscal  year.  AC 
and  USAR  funds  are  allocated  through  battalion  and  separate  com¬ 
pany.  In  the  National  Guard,  the  National  Guard  Bureau  makes 
allocations  to  state  headquarters.  Funds  are  allocated  based  on 
the  program  analysis  and  resource  review  (PARR)  and  COB  input. 
Unanticipated  changes  in  fund  priorities  affect  short-term 
planning . 
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The  proponent  for  TMACS  is  the  CG,  FORSCOM.  CG,  U.S.  Army 
Information  Systems  Software  Support  Command  is  the  responsible 
agency  for  TMACS  software. 

The  BTMS  is  the  standard  method  for  managing  training  in  units. 
(See  AR  350-1).  The  process  consists  of  the  planning,  resource, 
training,  and  evaluation  phases.  TMACS  is  used  in  both  the 
planning  and  resource  phases  of  the  BTMS.  The  following  is 
derived  from  FM25-2: 

a.  In  the  planning  phase,  the  unit  develops  long  and  short- 
range  training  plans.  The  plans  are  developed  from  training 
needs,  goals,  and  objectives  identified  in  regulations, 
specified  by  higher  commands,  or  determined  by  the  unit 
commander's  assessments  of  individual  and  collective  train¬ 
ing.  The  resulting  plans  are  separated  into  training 
events.  Each  training  event  is  entered  into  the  TMACS  using 
the  event  and  ammunition  worksheets  and  the  tables  and 
printouts  provided.  Training  events  are  identified  by 
training  event  category  cede'*  (TECCs)  ir.  the  TMACS  program. 
Each  training  event  is  considered  either  required  or 
optional.  Optional  training  events  are  given  a  priority 
based  on  the  commander's  guidance.  Ammunition  required  for 
each  training  event  is  entered  into  TMACS  in  conjunction 
with  the  training  event  from  the  ammunition  worksheet. 

b.  In  the  resource  phase,  whe  unconstrained  training  program 
developed  in  the  planning  stage  becomes  constrained  by  funds 
allocated  to  the  unit.  The  funding  limitations  for  fuel  and 
spare  parts  are  input  into  the  TMACS,  and  the  Optimum  Events 
Listing  is  printed.  This  printout  will  list  resourced 
events  and  then  nonresourced  events.  The  unit  commander  or 
training  officer  optimizes  the  training  program  by  using  the 


E3-C-3 


computer  to  adjust  the  projected  numbers  of  vehicles,  equip¬ 
ment,  estimated  mileage,  and  to  obtain  revised  training 
events  and  reallocated  resources.  This  process  makes  funds 
available  for  training  events  formerly  on  the  nonresourced 
list.  When  the  commander  decides  that  the  best  training 
program  is  available  relative  to  resources,  the  training 
program  is  passed  to  higher  headquarters  on  a  summary  disk. 
The  training  programs  required  but  nonresourced  are  for¬ 
warded  to  higher  headquarters  as  unfinanced  requirements. 
Higher  headquarters  may  either  reallocate  funds,  cancel  part 
of  the  training  requirements  for  that  unit,  or  cancel  the 
entire  requirement. 

c.  During  the  training  phase,  actual  records  of  resources  used 
are  entered  into  TMACS  to  develop  a  historical  data  base.  A 
summary  report  (the  Closure  Report)  is  an  accurate  review  of 
the  resources  used  to  complete  each  training  event.  The 
data  from  the  Closure  Report  is  input  to  the  TMACS  as 
accomplished  data.  This  process  also  coincides  with  the 
Execution  Budget  phase. 

d.  The  evaluation  phase  reviews  the  accomplished  data  in  the 
data  base  and  compares  it  to  the  planned  events,  the  annual 
budget  execution  records,  and  the  revalidation  or  change  of 
equipment  cost  factors.  This  process  assists  in  preparing 
future  training  programs. 

Using  the  TMACS  in  preparing  the  training  schedule  of  the  command 

operating  budget 

a.  Projection  of  Program  2  Mission  (Training)  dollar  require¬ 
ments  take  place  at  the  battalion  or  separate  company  level, 
is  consolidated  at  successive  headquarters,  and  is  submitted 
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t  o  HQ  DA .  it  represents  the  comma n  Jei  s  '  estimate  of  the 
resources  required  to  perform  the  training  mission. 

b.  Annual  instructions  for  preparing  the  training  portion  of 
the  COB  (Schedule  32  Instructions)  are  forwarded  to  MACOMs 
by  HQDA .  MACOMs  supplement  the  instructions  and  pass  them 
on  to  subordinate  commands  for  completion.  In  providing  the 
data  required  by  the  instructions,  units  use  the  software 
available  in  the  TMACS  for  a  portion  of  the  requirements. 
Completed  schedules  are  forwarded  through  channels  and 
compiled  by  MACOMs.  MACOMs  forward  summarized  Schedule  32 
Instructions  to  HQDA,  where  the  data  are  used  in  the  program 
objective  memorandum  (POM)  cycle. 

c.  To  prepare  the  training  schedule  of  the  COB,  the  commander 
must  first  complete  the  yearly  training  plan  as  is  done  in 
the  planning  phase  of  the  Battalion  Training  Management 
System  ( BTMS ) .  Then,  summaries  of  data  can  be  produced  for 
completing  the  Schedule  32  Instructions. 
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